Pure candle embeddings for Rust
Run dense, sparse, multi-vector and reranking models inside your Rust process, implemented on candle with no ONNX runtime. Every model in the catalog is checked against its reference implementation, with a status for each device and dtype.
cargo add canddingor candding = "0.1" under [dependencies]
Next on the roadmap: image models, then the Constella retrieval family.
Scores computed by candding with BAAI/
Pick a query
Wrap the HashMap in a Mutex, or use DashMap when many threads write at once.
0.781Measure throughput from a release build; a debug build leaves your own crate unoptimized.
0.527Cosine similarity is the dot product of two vectors after each is scaled to unit length.
0.511Embeddings turn text into vectors so that similar meanings land close together.
0.492Apple Metal runs the same forward pass as the CPU; only the attention kernel differs.
0.438Mean pooling averages the token states, weighted by the attention mask so padding is ignored.
0.394
Cosine similarity of the L2-normalized vectors, in F32 on the CPU, from just docs-demo at commit 2b10fcf. How these numbers are verified
//! Embed two passages and a query on the auto-selected device and score them.
use candding::TextEmbedding;
fn main() -> candding::Result<()> {
let model = TextEmbedding::builder("BAAI/bge-small-en-v1.5").build()?;
let passages = model.passage_embed(
&[
"Embeddings map text to vectors.",
"Rust is a systems programming language.",
],
None,
)?;
let query = model.query_embed(&["what do embeddings do"], None)?;
for (i, passage) in passages.iter().enumerate() {
let score: f32 = query[0].iter().zip(passage).map(|(q, p)| q * p).sum();
println!("passage {i}: {score:.4}");
}
Ok(())
}Every row in the catalog carries a status for each device and dtype, from a golden run against the model's own reference implementation, which the testing page names for every output: sentence-transformers, FlagEmbedding, fastembed, colbert-ai, transformers, or the model card's or the repository's own code. Image models and the Constella retrieval family are next on the roadmap.