Sparse output
What a sparse embedding is, what each of the six sparse heads indexes and computes, how two rows are scored, and which dtypes each head supports.
A sparse embedding is a row of (index, weight) pairs over a model's own output space instead of a fixed-width vector. Indices are ascending and unique, a repeated index keeps its largest weight, and only an exact zero is dropped. Weights can be negative, since MiniCoil applies a hyperbolic tangent.
pub struct SparseEmbedding {
pub indices: Vec<u32>,
pub values: Vec<f32>,
}let model = candding::SparseTextEmbedding::builder("prithivida/Splade_PP_en_v1").build()?;
let query = model.query_embed(&["a quick fox"], None)?;
let passage = model.passage_embed(&["The quick brown fox jumps over the lazy dog."], None)?;
let score = query[0].dot(&passage[0]);query_embed and passage_embed return one SparseEmbedding per text, and embed is the passage side. Sparse entries live in their own catalog, registry::sparse, and an id can sit in the dense catalog too, one set of weights behind two rows.
The six heads
| Head | Catalog id | Encoder | What an index names | Width |
|---|---|---|---|---|
SpladeMlm | prithivida/Splade_PP_en_v1 | BERT | an English WordPiece id | 30522 |
M3Linear | BAAI/bge-m3 | XLM-RoBERTa | an XLM-RoBERTa SentencePiece id | 250002 |
SpladeTied | naver/splade-code-06B | Qwen3, bidirectional | a Qwen3 byte-level BPE id | 151936 |
MiniCoil | jinaai/jina-embeddings-v2-small-en, with the head files of Qdrant/minicoil-v1 | JinaBERT | four components per word of a private vocabulary, or a hash for a word outside it | 76500, plus a disjoint hash range |
Bm42 | sentence-transformers/all-MiniLM-L6-v2 | BERT | a hash of a stemmed whole word | unbounded |
Bm25 | Qdrant/bm25 | none | a hash of a stemmed token | unbounded |
| Head | Passage weights | Query weights |
|---|---|---|
SpladeMlm | the masked-language-model logits, relu then log1p, maximum over the sequence | as the passage |
M3Linear | a relu'd Linear(hidden, 1) per position, scattered to its token id, special tokens dropped | as the passage |
SpladeTied | as SpladeMlm, with the backbone's embed_tokens as the decoder | as the passage |
MiniCoil | per whole word, stopwords and punctuation dropped: a known word's averaged states through its (512, 4) head slice, tanh and L2-normalized, an unknown word hashed; each scaled by a BM25-style term-frequency saturation | the encoder runs, and every term has weight 1.0 |
Bm42 | the last layer's attention from the first position, averaged over heads, summed per word, its maximum per stem rescored as ln(1 + value)^0.5 | no encoder: each unique stem hashed at 1.0 |
Bm25 | the term-frequency half of BM25 per stem | each unique stem at 1.0; see BM25 |
Bm42readssentence-transformers/all-MiniLM-L6-v2's weights, never the ONNX exportQdrant/all_miniLM_L6_v2_with_attentions, and its English stopword list is compiled into the crate.MiniCoilfetches its per-word matrix, vocabulary and stopwords fromQdrant/minicoil-v1at a pinned revision, and hashes unknown words differently fromBm42andBm25.
| Non-zero entries per row, nine fixture inputs | SpladeMlm | M3Linear | SpladeTied | MiniCoil | Bm42 | Bm25 |
|---|---|---|---|---|---|---|
| range | 7 to 177 | 0 to 95 | 0 to 3422 | 0 to 183 | 0 to 49 | 0 to 57 |
Bm25: computed, not encoded
Qdrant/bm25 has no encoder, no device and no dtype, so SparseTextEmbeddingBuilder::build refuses it before any fetch and names its own entry point, Bm25TextEmbedding::builder(), in the error. That builder takes no settings and cannot fail:
let model = candding::Bm25TextEmbedding::builder().build();
let passage = model.passage_embed(&["The quick brown fox jumps over the lazy dog."]);
let query = model.query_embed(&["a quick fox"]);Its registry entry reads SparseComputation::Computed where every other reads Encoded; the BM25 page has its usage and tokenizer.
Scoring two rows
SparseEmbedding::dot is the scoring rule the references declare: the sum of products over the indices both rows share, zero when either is empty. Neither side is normalized, so a score is not bounded to [-1, 1], and a Bm25 row carries only the term-frequency half of BM25.
Edge cases
| Case | Behaviour |
|---|---|
an empty string on naver/splade-code-06B, whose tokenizer adds no special tokens | an empty row, alone or in a batch, where its reference raises a reshape error |
an input whose only tokens are special ones, on M3Linear | an empty row |
| batching | naver/splade-code-06B's single and batched rows agree within 2e-4; no other encoded head has a batch tolerance test |
Verification and dtypes
Each encoded head's golden suite compares every fixture row with its reference, and in F32 requires a Jaccard of 1.0, the same non-zero indices, besides the values. In half precision, each cell below is the worst max-abs difference against a ceiling of 5e-3 in F16 and 5e-2 in BF16, with the lowest Jaccard when it falls below 1.0:
| Head | F16, CPU | F16, Metal | BF16, Metal |
|---|---|---|---|
SpladeMlm | unsupported, 3.3e-2 | unsupported, 6.7e-3, Jaccard 0.96 | unsupported, 5.2e-2, Jaccard 0.93 |
M3Linear | unsupported, 2.7e-2 | verified, 1.2e-3 | verified, 3.4e-2 |
SpladeTied | unsupported, 1.2e-1, Jaccard 0.97 | unsupported, 9.5e-3, Jaccard 0.99 | unsupported, 7.1e-2, Jaccard 0.97 |
MiniCoil | unsupported, 1.3e-2 | verified | verified |
Bm42 | verified, 9.4e-4 | verified, 1.9e-4 | verified, 1.5e-3 |
A CPU figure for a refused dtype is the first fixture that missed, so it is a lower bound. The ceilings come from the references in half precision: SPLADE's from sentence-transformers' SparseEncoder end to end, bge-m3's from its head cast alone; naver/splade-code-06B has no half-precision reference of its own and takes the same ceilings.
Model files and cache
Where candding looks for weights, which files it downloads and reads, how gated repositories work and how to use a local directory.
Multi-vector output
What a multi-vector embedding is, the conventions its four checkpoints disagree on, late-interaction scoring, and each checkpoint's status per dtype.