candding
Concepts

Multi-vector output

What a multi-vector embedding is, the conventions its four checkpoints disagree on, late-interaction scoring, and each checkpoint's status per dtype.

A multi-vector embedding is a matrix with one row per surviving token instead of one pooled vector. A query and a document are compared by late interaction, MaxSim: for each query row, the largest dot product against any document row, summed over the query rows.

let model = candding::MultiVectorTextEmbedding::builder("colbert-ir/colbertv2.0").build()?;
let query = model.query_embed(&["a quick fox"], None)?;
let passage = model.passage_embed(&["The quick brown fox jumps over the lazy dog."], None)?;
let descriptor = model.descriptor();
let score = candding::multi_vector::late_interaction_score_for(
  descriptor.score_reduction,
  &query[0],
  &passage[0],
  descriptor.masked_document_row_fill,
);

query_embed and passage_embed return one MultiVectorEmbedding per text, since a checkpoint can tokenize and pad a query differently from a document; embed is the document side. Every row is L2-normalized, unless the builder's normalize(false) or the CLI's --no-normalize asks for the rows as the projection writes them. The entries live in their own catalog, registry::multi_vector.

The four checkpoints

CheckpointEncoderHeadProjectionBiasKeeps the first rowScore reductionQuery rowsQuery attends to its mask tokens
BAAI/bge-m3XLM-RoBERTaM3Colbert1024 to 1024yesnoMeanByQueryLengthits own lengthno
colbert-ir/colbertv2.0BERTStanfordColbert768 to 128noyesSum32no
answerdotai/answerai-colbert-small-v1BERTStanfordColbert384 to 96noyesSum32no
jinaai/jina-colbert-v2XLM-RoBERTa (flash)JinaColbert1024 to 128noyesSum32yes

Each convention is descriptor data, keeps_cls_row, score_reduction and attends_to_mask_tokens, not a rule in the head or the scoring function. BAAI/bge-m3's reference divides the MaxSim sum by the query's real row count, so the wrong reduction scales a score by that count.

vidore/colpali-v1.3 is on the roadmap, with the image models.

Query padding, markers and punctuation

The three ColBERT-style checkpoints pad every query to exactly 32 rows with the mask token, [MASK] for the BERT pair and <mask> for jinaai/jina-colbert-v2, read from tokenizer_config.json; the 32 is the checkpoint's contract, not a max_length. Each document keeps its own row count up to the truncation cap.

CheckpointsQuery markerDocument marker
colbert-ir/colbertv2.0, answerdotai/answerai-colbert-small-v1[unused0] [unused1]
jinaai/jina-colbert-v2[QueryMarker] [DocumentMarker]

All three drop punctuation rows from documents, so the surviving rows are not a prefix of the sequence. BAAI/bge-m3 has no fixed query length, no markers and no punctuation skiplist.

Late-interaction scoring

multi_vector::late_interaction_score takes an optional mask per side, None meaning every row is real, as every catalog checkpoint returns; late_interaction_score_for applies a checkpoint's own ScoreReduction and masked_document_row_fill.

Edge caseScore
a document with no real rows-1e9, so it sorts last; never rescaled by the reduction
a query with no real rows against a real document0.0
a masked document rowfilled with the descriptor's MaskedRowFill before the maximum: the dtype minimum for the ColBERT-style three, zero for BAAI/bge-m3
a masked query rowleft out of the sum

No shape panics. BAAI/bge-m3's multi-vector head tokenizes with the published tokenizer.json; the XLM-RoBERTa page has the trailing-whitespace case where that differs from FlagEmbedding.

Verification

Each checkpoint's golden suite scores all 81 query-document pairs of the nine fixture inputs against its reference. F32 is verified on the CPU and on Metal for all four, and on CUDA for all but jinaai/jina-colbert-v2, which no workflow fetches; the catalog shows each dtype's CUDA cell.

CheckpointF16BF16
BAAI/bge-m3unsupported: cosine 0.9964 on the CPU and 0.9959 on Metal, against a 0.998 floorunsupported: cosine 0.9939 on Metal, against a 0.995 floor
colbert-ir/colbertv2.0, answerdotai/answerai-colbert-small-v1, jinaai/jina-colbert-v2verified on the CPU and on Metalverified on Metal

BF16 is refused on the CPU for all four, as for every model. BAAI/bge-m3's sparse head passes F16 on Metal on the same encoder; the multi-vector head's miss has no established cause.

BAAI/bge-m3 has a dense, a sparse and a multi-vector entry, one set of weights behind three rows with their own statuses. The other three checkpoints are multi-vector entries only, so the catalog's multi-vector table lists them and the family pages' model tables do not.

On this page