Pipeline
The stages between a text and a vector, the tensor shapes between them, length-sorted batching and the effective maximum length.
One call to embed, query_embed or passage_embed runs the same stages for every dense model on every backend. A family supplies only the forward pass; templates, pooling, truncation and normalization are written once, outside family code.
Stages
| Stage | What happens | Shape after it |
|---|---|---|
| Template | the query or passage template, its {text} and {instruction} replaced once, so placeholders inside the text stay; embed applies none | text |
| Tokenizer | the published tokenizer.json, with right padding and truncation to the effective max length, special tokens included | (batch, seq) ids, mask and token type ids, seq the chunk's longest row |
| Forward | the family's Encoder implementation | (batch, seq, hidden) |
| Pooling | cls, mean or last_token, per the descriptor; see pooling and templates | (batch, hidden) |
| Upcast | the pooled tensor becomes F32, whatever dtype the weights are in | (batch, hidden) |
| Dense modules | the repository's sentence-transformers Dense modules in modules.json order, each a linear layer with an optional bias and an identity or tanh activation; only google/embeddinggemma-300m in the catalog has them | (batch, dim) |
| LayerNorm after pooling | zero mean and unit variance per row, with no affine parameters, when the descriptor sets layer_norm_after_pooling, as only nomic-ai/nomic-embed-text-v1.5 does | (batch, dim) |
| Matryoshka truncation | the *_dim methods keep the first dim columns, on a model whose descriptor allows it | (batch, dim) |
| Normalization | each row divided by its L2 norm, clamped at 1e-12 | (batch, dim) |
| Order restore | the rows return to the caller's order, one Vec<f32> per input text | Vec<Vec<f32>> |
Normalization runs unless a descriptor or normalize(false) turns it off, as pooling and templates explains. The sparse, multi-vector and rerank models replace everything after the forward pass with their own head.
Batching
The last argument of every embedding method is the batch size: None uses the builder's default of 32. Inputs are sorted by token length, longest first, and each chunk is padded only to its own longest row; the output is still in input order, and an empty input list returns an empty vector without running the model.
One case makes a vector depend on its batch beyond rounding: on nomic-ai/nomic-embed-text-v1 and Snowflake/snowflake-arctic-embed-m-long, an input past the trained length rescales the rotary table for every row of its batch; see NomicBERT.
Effective max length
| Model | Limit |
|---|---|
| a registry entry | the builder's max_length when set, otherwise the descriptor's default, the catalog's Max length column |
| a repository outside the registry, with no descriptor | the builder's max_length when set, otherwise its sentence_bert_config.json max_seq_length, otherwise 512 |
| either | capped at the model's position limit; max_length() reports the value in use |
The limit counts special tokens, so BERT's [CLS] and [SEP] leave 510 content tokens under a limit of 512, and token_count reports the tokens that survive truncation. A default can be raised up to the position limit, at an attention memory quadratic in the length, but the vectors then no longer match what the repository publishes.
Where families plug in
Each architecture is one implementation of one trait:
pub struct EncoderInput<'a> {
pub input_ids: &'a Tensor,
pub attention_mask: &'a Tensor,
pub token_type_ids: &'a Tensor,
pub lengths: &'a [usize],
}
pub trait Encoder: Send + Sync {
fn forward(&self, input: &EncoderInput) -> Result<Tensor>;
fn hidden_size(&self) -> usize;
fn max_position_embeddings(&self) -> usize;
fn weights_dtype(&self) -> DType;
}config.json's model_type selects the implementation at load, so a repository outside the registry with a supported architecture loads without an entry. Porting a model has the checklist.