Rust · candle · Metal · CUDA

Pure candle embeddings for Rust

Text embedding models on candle with no ONNX: dense, sparse, late-interaction, reranker and image models on CPU, Apple Metal and NVIDIA CUDA, verified against sentence-transformers.

Get startedModel catalogGitHub
13
verified models
3
backends from one code path
130
golden tests against sentence-transformers
candding/examples/quickstart.rs
//! Embed two passages and a query on the auto-selected device and score them.

use candding::TextEmbedding;

fn main() -> candding::Result<()> {
  let model = TextEmbedding::builder("BAAI/bge-small-en-v1.5").build()?;
  let passages = model.passage_embed(
    &[
      "Embeddings map text to vectors.",
      "Rust is a systems programming language.",
    ],
    None,
  )?;
  let query = model.query_embed(&["what do embeddings do"], None)?;
  for (i, passage) in passages.iter().enumerate() {
    let score: f32 = query[0].iter().zip(passage).map(|(q, p)| q * p).sum();
    println!("passage {i}: {score:.4}");
  }
  Ok(())
}

No ONNX anywhere

Every architecture is written on candle-core and candle-nn in this crate. No runtime downloads, no exported graphs, no candle-transformers dependency.

One code path, three backends

Model code has no backend branches. CPU, Apple Metal and NVIDIA CUDA run the same forward; only the attention kernel switches.

Verified, not assumed

Each registered model passes golden tests against sentence-transformers references: cosine at least 0.9999 and max-abs at most 1e-4 in F32, plus batch, norm, truncation and STS checks.

The fastembed surface and beyond

The model list follows fastembed and adds the decoder families it lacks, starting with Qwen3-Embedding and EmbeddingGemma on the roadmap.

Templates and Matryoshka built in

Query and passage templates, instruction prompts and dimension truncation are descriptor data, applied the same way the reference implementation applies them.

A CLI for the loop

Fetch weights through the Hugging Face CLI, embed from the shell, print token counts, and benchmark batch sizes on any device.