Rust · candle · Metal · CUDA
Pure candle embeddings for Rust
Text embedding models on candle with no ONNX: dense, sparse, late-interaction, reranker and image models on CPU, Apple Metal and NVIDIA CUDA, verified against sentence-transformers.
//! Embed two passages and a query on the auto-selected device and score them.
use candding::TextEmbedding;
fn main() -> candding::Result<()> {
let model = TextEmbedding::builder("BAAI/bge-small-en-v1.5").build()?;
let passages = model.passage_embed(
&[
"Embeddings map text to vectors.",
"Rust is a systems programming language.",
],
None,
)?;
let query = model.query_embed(&["what do embeddings do"], None)?;
for (i, passage) in passages.iter().enumerate() {
let score: f32 = query[0].iter().zip(passage).map(|(q, p)| q * p).sum();
println!("passage {i}: {score:.4}");
}
Ok(())
}No ONNX anywhere
Every architecture is written on candle-core and candle-nn in this crate. No runtime downloads, no exported graphs, no candle-transformers dependency.
One code path, three backends
Model code has no backend branches. CPU, Apple Metal and NVIDIA CUDA run the same forward; only the attention kernel switches.
Verified, not assumed
Each registered model passes golden tests against sentence-transformers references: cosine at least 0.9999 and max-abs at most 1e-4 in F32, plus batch, norm, truncation and STS checks.
The fastembed surface and beyond
The model list follows fastembed and adds the decoder families it lacks, starting with Qwen3-Embedding and EmbeddingGemma on the roadmap.
Templates and Matryoshka built in
Query and passage templates, instruction prompts and dimension truncation are descriptor data, applied the same way the reference implementation applies them.
A CLI for the loop
Fetch weights through the Hugging Face CLI, embed from the shell, print token counts, and benchmark batch sizes on any device.