Pooling and templates
The three pooling modes, the query and passage templates each catalog model carries, instructions, and Matryoshka truncation.
Pooling
An encoder returns one hidden state per token; pooling turns those into the single vector a retrieval system stores. The mode is descriptor data, and getting it wrong produces vectors that look healthy and rank badly, which is why every catalog model is checked against reference vectors that were pooled the same way.
| Pooling | The vector is | Catalog models |
|---|---|---|
cls | the hidden state of token 0 | BAAI/bge-small-en-v1.5, BAAI/bge-base-en-v1.5, BAAI/bge-large-en-v1.5, BAAI/bge-small-zh-v1.5, mixedbread-ai/mxbai-embed-large-v1, Snowflake/snowflake-arctic-embed-xs, Snowflake/snowflake-arctic-embed-s, Snowflake/snowflake-arctic-embed-m, Snowflake/snowflake-arctic-embed-l |
mean | the attention-mask weighted average over the real tokens | sentence-transformers/all-MiniLM-L6-v2, thenlper/gte-base, thenlper/gte-large, sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 |
last_token | the hidden state at index length - 1 | none in the catalog yet |
Mean pooling multiplies the hidden states by the mask, sums over the sequence and divides by the mask sum clamped at 1e-9, in the dtype the hidden states are in, exactly as sentence-transformers does it. Last-token pooling reads the length that came out of the attention mask instead of taking the final column, because padding is always on the right; it is the mode the decoder-based embedders such as Qwen3 need, and the code for it is in place ahead of that family.
A repository outside the registry that brings no descriptor gets its pooling from its own 1_Pooling/config.json: the CLS flag wins, then the last-token flag, and mean is the fallback.
Templates
Query and passage templates are descriptor data too, with {text} and {instruction} as the only placeholders. query_embed applies the query template, passage_embed applies the passage template, and embed applies neither; the golden references hold one set of vectors per path, so all three are checked. A missing prefix costs retrieval quality without changing anything you can see in the output, so the templates ship with the model rather than with your call site.
| Models | Query template | Passage template |
|---|---|---|
| bge English (small, base, large), mxbai-embed-large-v1, Snowflake Arctic (xs, s, m, l) | Represent this sentence for searching relevant passages: {text} | none |
| BAAI/bge-small-zh-v1.5 | 为这个句子生成表示以用于检索相关文章：{text} | none |
| all-MiniLM-L6-v2, gte-base, gte-large, paraphrase-multilingual-MiniLM-L12-v2 | none | none |
For the models with no templates at all, embed, query_embed and passage_embed return the same vectors. The families on the roadmap bring the other shapes.
| Family | Templates |
|---|---|
| e5 | query query: {text}, passage passage: {text} |
| nomic | query search_query: {text}, passage search_document: {text} |
| Qwen3 | query Instruct: {instruction}\nQuery: {text} |
| EmbeddingGemma | query task: search result | query: {text}, passage title: none | text: {text} |
Instructions
Instruct-style models put a task description in front of the query, and the descriptor carries a default for it. query_embed_with_instruction replaces that default for one call, filling the {instruction} placeholder while {text} still takes the query. No catalog model has an instruction placeholder, so the method exists for the instruct families rather than for BERT.
Matryoshka
embed_dim, query_embed_dim and passage_embed_dim take a dimension and keep that many leading columns. Truncation happens before L2 normalization; when Dense modules arrive with EmbeddingGemma they slot in between pooling and truncation, which is also where the pipeline page will list them.
Two errors guard the call: a model whose descriptor does not declare Matryoshka support returns CanddingError::MrlUnsupported before any tokenization happens, and a dimension of zero or one greater than the model's own dimension returns CanddingError::InvalidDim. None of the catalog models supports Matryoshka truncation; Qwen3-Embedding and EmbeddingGemma do, and the descriptors will declare it when those families land.