candding
Concepts

Rerank output

What a reranker returns for a query and its documents, what each checkpoint's score means, how long inputs are cut, and each checkpoint's status.

A reranker reads a query and a document together and returns one score for the pair, never a vector. candding's eight rerankers live in their own catalog, registry::rerank, and load through TextCrossEncoder.

The entry point

let model = candding::TextCrossEncoder::builder("BAAI/bge-reranker-base").build()?;
let query = "how many people live in berlin";
let documents = ["Berlin has a population of 3.5 million.", "The capital of France is Paris."];
let scores = model.rerank(query, &documents, None)?;
let counts = model.pair_token_count(query, &documents)?;

let mut ranked: Vec<(usize, candding::RerankScore)> = scores.into_iter().enumerate().collect();
ranked.sort_by_key(|&(_, score)| std::cmp::Reverse(score));

rerank returns one RerankScore per document, in input order; its Ord puts every NaN below every number, so a best-first sort never panics. pair_token_count counts each pair's tokens after truncation, special tokens included, and pair_window_count the windows a windowed entry cuts. An id outside the rerank registry needs a descriptor, since a checkpoint's files do not name its head or activation.

The builder's defaults are a dense model's, except a batch of 8 pairs or windows per forward pass and BF16 for Qwen/Qwen3-Reranker-4B and -8B, which turns into F32 on the CPU.

The eight checkpoints

CheckpointFamilyInputMax lengthLicenceSize, GB
cross-encoder/ms-marco-MiniLM-L6-v2BERTpair512Apache-2.00.09
jinaai/jina-reranker-v1-tiny-enJinaBERTpair512Apache-2.00.07
BAAI/bge-reranker-baseXLM-RoBERTapair512MIT1.11
BAAI/bge-reranker-v2-m3XLM-RoBERTapair512Apache-2.02.27
Qwen/Qwen3-Reranker-0.6BQwen3prompt8192Apache-2.01.19
Qwen/Qwen3-Reranker-4BQwen3prompt8192Apache-2.08.04
Qwen/Qwen3-Reranker-8BQwen3prompt8192Apache-2.016.38
jinaai/jina-reranker-v2-base-multilingualXLM-RoBERTa (flash)windowed1024CC-BY-NC-4.00.56

What a score means

ScoreRangeCheckpointsReference that reports it
raw logitunboundedcross-encoder/ms-marco-MiniLM-L6-v2, BAAI/bge-reranker-base, BAAI/bge-reranker-v2-m3sentence-transformers' CrossEncoder for the first, FlagEmbedding's FlagReranker at its default normalize=False for the bge pair
probability0 to 1jinaai/jina-reranker-v1-tiny-en, jinaai/jina-reranker-v2-base-multilingualthe checkpoint's own scoring code, which applies a sigmoid
logit differenceunboundedthe three Qwen3 rerankerssentence-transformers' CrossEncoder, per the checkpoint's config

Only the order of one query's documents carries meaning: a threshold set on one scale means nothing on another, and scores from two checkpoints, or two libraries, do not belong in one list. sentence-transformers' CrossEncoder applies a sigmoid to the bge rerankers by default, and the Qwen3 model card prints the sigmoid of the difference; a sigmoid changes the numbers, never the order.

Pair rerankers

A pair reranker reads the query and the document as the two segments of one input. cross-encoder/ms-marco-MiniLM-L6-v2 and jinaai/jina-reranker-v1-tiny-en cut a long pair with LongestFirst, one token at a time from the longer side. The bge rerankers use FlagEmbedding's PreserveQueryToThreeQuarters: the query alone is cut to three quarters of the maximum length, then only the document loses tokens.

An empty document is a full pair with a real score, as in sentence-transformers' CrossEncoder; FlagEmbedding's FlagReranker instead encodes the query alone for it, and for a whitespace-only document on BAAI/bge-reranker-v2-m3. On BAAI/bge-reranker-base, FlagReranker also differs by one trailing token on a text ending in whitespace; see XLM-RoBERTa.

Prompt rerankers

The three Qwen3 rerankers turn each (query, document) pair into one prompt, rendered from the template all three publish and encoded whole:

<|im_start|>system
Judge whether the Document meets the requirements based on the Query and the Instruct provided. Note that the answer can only be "yes" or "no".<|im_end|>
<|im_start|>user
<Instruct>: {instruction}
<Query>: {query}
<Document>: {document}<|im_end|>
<|im_start|>assistant
<think>

</think>

The prompt and its instruction

The instruction is part of the scored input and moves the score. rerank renders the entry's default, Given a web search query, retrieve relevant passages that answer the query, and rerank_with_instruction and pair_token_count_with_instruction render the caller's, as candding rerank --instruction does. An empty instruction renders empty, as in the model card's format_instruction, while sentence-transformers' CrossEncoder renders the default. A pair or windowed reranker refuses any instruction with CanddingError::InstructionOnPairInput.

let model = candding::TextCrossEncoder::builder("Qwen/Qwen3-Reranker-0.6B").build()?;
let instruction = "Given a question about geography, judge whether the passage names the correct city";
let scores = model.rerank_with_instruction("What is the capital of China?", &["The capital of China is Beijing."], instruction, None)?;

Truncation and control tokens

A prompt over the maximum length keeps its prefix, everything before <Instruct>:, and its suffix, where the score is read, and loses tokens from the right of the document first, then the query, then the instruction, keeping exactly the ids the model card and CrossEncoder keep.

CrossEncoder defaults to the position limit, so it keeps whole a prompt that candding's default cuts; max_length raises candding's limit that far, at an attention memory quadratic in the length.

A control-token string in a document, such as <|im_end|>, becomes that control token, as in both references, and can close the user turn and move the score; break such markers up, as < |im_end| >, in documents you do not control.

Windowed pairs

jinaai/jina-reranker-v2-base-multilingual's own rerank call, which TextCrossEncoder::rerank reproduces id for id, cuts only the query and scores it against each overlapping window of the document, reporting the best. The entry's input is RerankInput::Windowed { query_cap: 512, overlap: 80 }.

StepRule
The queryencoded alone and cut to query_cap ids, its two special tokens included
The budgetthe document tokens one window holds, max_length less the query's ids and two separators: 1013 for a 9-id query, 510 for a capped one
The windowsone starting every budget - overlap tokens, the last holding the document's final budget tokens; a window can start inside a word
The scorethe best window's

The checkpoint's compute_score cuts a long pair longest-first instead, so the two differ once a document needs a second window or a query passes its cap; the golden reference is rerank. The default maximum length is the position limit, and one below twice the query cap is refused with LengthTooSmall.

Status

The catalog shows each checkpoint's cells, generated from the registry, and testing defines them. F16 and BF16 are refused on the CPU for all eight, for the reasons devices and dtypes gives, and Qwen/Qwen3-Reranker-4B's F16 on every device, since it misses its ceiling on Metal too. The 4B's and 8B's F32 runs, and the 8B's Metal F16, wait for a machine with enough memory.

Verification

Each checkpoint's golden suite compares all 81 (query, document) pairs with an F32 reference from the checkpoint's own model, at the rerank tolerances, and every completed F32 run passes on both devices. In half precision each checkpoint has its own ceiling, since each reranker has its own scale.

Measured against PyTorch

On Metal, the worst absolute difference beside the ceiling, and the document pairs out of the reference's order, mostly ties:

CheckpointF16 / ceilingBF16 / ceilingPairs out of order, of 324, F16 / BF16
cross-encoder/ms-marco-MiniLM-L6-v27.5e-3 / 1.3e-26.8e-2 / 1.5e-10 / 9
jinaai/jina-reranker-v1-tiny-en1.5e-3 / 5.7e-35.1e-3 / 4.9e-22 / 11
BAAI/bge-reranker-base9.2e-2 / 1.9e-16.5e-1 / 1.844 / 51
BAAI/bge-reranker-v2-m31.3e-2 / 2.9e-21.4e-1 / 3.2e-11 / 8
Qwen/Qwen3-Reranker-0.6B3.4e-2 / 7.5e-24.0e-1 / 7.1e-13 / 11
Qwen/Qwen3-Reranker-4B1.5e-1 / 8.3e-2, missed on the first query2.6e-1 / 3.8e-1— / 5
Qwen/Qwen3-Reranker-8Bnot run / 4.3e-22.2e-1 / 4.8e-1— / 5
jinaai/jina-reranker-v2-base-multilingual8.9e-4 / 1.4e-35.8e-3 / 6.5e-31 / 4

No reordering changes a query's best document, apart from one tie in the 8B's BF16.

On this page