Rerank output

A reranker scores a query against a document. The two texts are encoded together as one pair, a single sequence with the checkpoint's own separator tokens between them, and the model returns one number for that pair, never a vector. candding ships this as its fourth output kind, beside dense, sparse and multi-vector, for four checkpoints on encoder families it already runs.

let model = candding::TextCrossEncoder::builder("BAAI/bge-reranker-base").build()?;
let query = "how many people live in berlin";
let documents = ["Berlin has a population of 3.5 million.", "The capital of France is Paris."];
let scores = model.rerank(query, &documents, None)?;
let counts = model.pair_token_count(query, &documents)?;

let mut ranked: Vec<(usize, candding::RerankScore)> = scores.into_iter().enumerate().collect();
ranked.sort_by_key(|&(_, score)| std::cmp::Reverse(score));

rerank(query, &documents, batch_size) returns one RerankScore per document, in the order the documents were given, never sorted by relevance. RerankScore::value() is the f32 itself, and RerankScore implements Ord as a total order that puts every NaN, of either sign, below every number and orders numbers as f32::total_cmp does, so sorting scores never panics and a best-first sort ranks a NaN last on every platform; the fragment above sorts best first with Reverse and keeps each document's input position beside its score. pair_token_count returns how many tokens each pair encodes to after truncation, query and special tokens included, from the same encoding rerank runs. An id outside the rerank registry is refused with CanddingError::UnregisteredRerankModel unless the builder is handed a descriptor, because nothing in a checkpoint's files says which head or which activation its reference uses.

The four checkpoints

All four sit in registry::rerank alone, with no dense catalog row; the catalog lists them in its rerank table.

What a caller receives

The number a RerankScore carries is what the checkpoint's own reference reports, and the references disagree. jinaai/jina-reranker-v1-tiny-en's own compute_score applies a sigmoid, so its score is a probability between zero and one; the other three report the raw logit of the head's final projection. The activation is per-checkpoint registry data, ScoreActivation::Identity or ScoreActivation::Sigmoid, applied once by the head. candding never adds or removes one, and candding describe prints it.

A sigmoid is monotonic, so within one model it never changes the order: jinaai/jina-reranker-v1-tiny-en's probabilities rank documents exactly as its logits would. It matters when a score is compared with anything outside that one model's ranking. A threshold picked on one scale means nothing on the other, and scores from two checkpoints, or from candding and a library that applies a different activation to the same checkpoint, cannot be mixed in one list. sentence-transformers' CrossEncoder is such a library for the two bge rerankers: for a single-output head with no declared activation it applies a sigmoid by default, where FlagReranker and candding return the logit. The golden suite checks each entry's ScoreActivation against the one its reference was generated with, and fails before comparing any score when the two differ.

Heads and truncation

Both heads compute the same function of the classification position, a Linear(hidden, hidden), a tanh and a Linear(hidden, 1). They differ in where a checkpoint keeps those weights and in which encoder sits under them.

PairTruncationPolicy decides what a pair over the maximum length gives up. LongestFirst is the tokenizer's own pair truncation: one token at a time from whichever side is longer, until the pair fits. PreserveQueryToThreeQuarters is FlagEmbedding's: the query alone is cut to three quarters of the maximum length, and only the document loses tokens after that, down to none when the query's share and the special tokens fill the budget. The share and the special tokens can exceed the maximum length only when it is at most four times one less than the special tokens, eight or less for BERT's three-token template and twelve or less for the four-token templates. There, a query longer than the maximum length minus the special tokens makes the pair longer than the maximum length, as the golden reference generator builds it; the default of 512 is far above that. A pair that fits the maximum length, with a query inside three quarters of it, loses nothing under either policy, so a port that truncates the pair as a unit matches FlagEmbedding on short inputs and diverges only on long ones; in the golden fixtures, those are the pairs with the long text on either side.

The counts leave out the special tokens.

Where candding and a live Python call disagree

Two differences between candding and a live call through a Python library are in the input, not in the arithmetic, and both are measured. Neither reaches the golden suite, whose references are built from the published tokenizer.json the way candding builds its own input.

The published tokenizer

For BAAI/bge-reranker-base, the rule the XLM-RoBERTa family page states for BAAI/bge-m3 holds: its published tokenizer.json, which candding replays, and the AutoTokenizer a live FlagReranker call tokenizes with disagree if and only if a text's normalized form ends in whitespace or is whitespace-only, always by one trailing token. On a corpus of prose, whitespace runs, Unicode spaces, multilingual text and code, they disagree on exactly the inputs where BAAI/bge-m3's file and AutoTokenizer disagree. BAAI/bge-reranker-v2-m3 publishes a different file, whose normalizer strips trailing whitespace, and on that corpus its file and AutoTokenizer agree on every input; a whitespace-only input normalizes to nothing in both, the case the next section covers.

An empty document

candding encodes an empty document as a full pair: the query, the separators and an empty second segment, scored like any other pair. A single-pair transformers tokenizer call with an empty second text reads it as no second text at all and returns a single-sequence encoding. FlagEmbedding's FlagReranker goes through that call: on transformers 5.16.1 its prepare_for_model_compat finds no prepare_for_model to call, decodes the query and the document back to text and tokenizes them as a pair, so an empty document collapses there too. For BAAI/bge-reranker-v2-m3 a whitespace-only document is an empty one, because its published file's Strip step normalizes it to nothing, so a live FlagReranker call collapses it as well while candding encodes the full pair. For cross-encoder/ms-marco-MiniLM-L6-v2, sentence-transformers' CrossEncoder builds the full pair and returns the golden reference's own score for it. The query below is the short fixture, The quick brown fox jumps over the lazy dog.

ALiBi slopes for jina-reranker-v1-tiny-en

jinaai/jina-reranker-v1-tiny-en runs on the JinaBERT family, whose only positional signal is ALiBi, and its twelve heads are not a power of two, so four of its slopes come from the interpolated tail. The six jina-embeddings-v2 checkpoints take their slopes from the shared jinaai/jina-bert-implementation code. This checkpoint bundles its own modeling_bert.py instead, and that file halves the interpolated tail after computing it. candding carries the difference as AlibiSlopeVariant::HalvedInterpolatedTail on this entry alone; every dense JinaBERT entry passes Canonical, and a head count that is a power of two gets the same slopes from either.

With the canonical slopes every score stayed finite and plausible, and an F32 column of 8e-2, set to clear the deviation before its cause was known, let it pass. The column is 1e-4 now.

Verification

Full precision is held to one column on both devices: an absolute difference of at most 1e-4 from the reference score on every one of the 81 pairs, and each query's nine documents in the reference's order, except documents the reference scores exactly equal. Half precision has no shared column. A checkpoint's F16 or BF16 ceiling is twice the worst deviation PyTorch itself shows at that dtype from its own F32 scores on the same 81 pairs, the larger of its CPU and MPS runs, because a reranker's score is an unnormalized number whose scale and half-precision sensitivity belong to the checkpoint: PyTorch's own BF16 deviation spans 2.2e-2 to 0.88 across these four. One shared ceiling is loose for one checkpoint and tight for another; the shared 5e-1 this rule replaced passed BAAI/bge-reranker-v2-m3's CPU F16 at a worst pair 28 times PyTorch's own, and failed BAAI/bge-reranker-base's Metal BF16, whose worst pair is smaller than PyTorch's. Every status below comes out the same for any factor strictly between two ratios of candding's worst pair to PyTorch's, 1.15304322 and 3.68466955: the largest on Metal, cross-encoder/ms-marco-MiniLM-L6-v2's at F16, and the smallest on the CPU at F16, jinaai/jina-reranker-v1-tiny-en's. A factor of 1.15304 already fails the first.

The CPU F16 figures come from runs made before those modules became refusals: an earlier sweep under the old shared ceiling for three checkpoints, and a diagnostic run for BAAI/bge-reranker-base. candding's CPU F16 misses every reranker's ceiling, at 3.7 to 27.9 times PyTorch's own deviation, where its Metal F16 stays within 1.153 times of it; candle's CPU F16 matmul accumulates in half precision, the backend fact the devices and dtypes page records. F16 is therefore accelerator-only for all four: the catalog cell reads unsupported on the CPU, the builder refuses it there with CanddingError::UnsupportedOnDevice, and the golden module passes on the CPU as a refusal check. BF16 is refused on the CPU for all four, as for every catalog entry that computes on a processor, because candle has no CPU BF16 matmul.

BF16 costs BAAI/bge-reranker-base more than any other checkpoint here, in both implementations: candding's worst pair on Metal is 6.5042233e-1, and PyTorch's own is 7.7151608e-1 on the CPU and 8.7698483e-1 on MPS, on reference scores that span -10.19688 to 10.30495. That clears its ceiling of 1.7539697, so BF16 is verified on Metal, and a BF16 score can still sit more than half a logit from the F32 reference; the same checkpoint's worst Metal F16 pair is 9.1765165e-2. Its F16 on the CPU misses its ceiling of 1.934228e-1 at 8.534839e-1, 8.8 times PyTorch's own F16 deviation, and is refused as accelerator-only. Its BF16 on Metal was marked unsupported on every device under the shared 5e-1 ceiling, which 6.5042233e-1 exceeds, and is verified under its own.

Half-precision scores are quantized to the dtype's step and can tie or swap documents whose scores are close, so the ranking check applies to F32 alone. On Metal, out of the 324 document pairs the nine queries rank:

Most of these are ties the half-precision score creates, and none changes a query's best document.

No cell has run on CUDA: no CUDA-capable machine has been available to this project and no workflow builds the cuda feature, so every CUDA cell reads untested. The testing page describes the golden_rerank suite and how its reference fixtures are generated.

Command line

candding rerank cross-encoder/ms-marco-MiniLM-L6-v2 "how many people live in berlin" "Berlin has a population of 3.5 million." "The capital of France is Paris." "" --device cpu

0	8.650594	Berlin has a population of 3.5 million.
2	-10.029929	
1	-11.319403	The capital of France is Paris.

Each line is a document's position on the command line, its score and the document, best first. The empty third document is a real pair with a real score, here above the unrelated sentence. The CLI page has the --json and --count forms and every flag.

Not here yet

Four of the phase's eight rerankers are not in the catalog. Each group below is a milestone of its own, and the roadmap lists all four.

• Qwen/Qwen3-Reranker-0.6B, -4B and -8B are decoders rather than encoders: they score by reading two columns of their own unembedding matrix, the yes and no logits at the last position, rather than through a head, behind a chat prompt with three slots where the existing template takes two. The 8B also carries an untied output matrix, on top of the memory problem that already leaves its embedding sibling's processor status open.
• jinaai/jina-reranker-v2-base-multilingual is non-commercial, which a machine check keeps out of the ci and metal workflows. Its encoder carries a learned position table that the XLM-RoBERTa (flash) family refuses, and it is the only one of the eight whose scoring is not one pair through one forward pass: its own method splits a long document into overlapping windows and keeps the highest score.

A fourth catalog, not a mode

Reranking is a fourth object graph beside the dense, sparse and multi-vector ones. RerankModel is its own trait with no query-side entry point: the tokenizer assembles query and document into one pair before the model sees either, so one forward pass scores the pair.