Porting a model

Adding a model to candding is a checklist with a pass or a fail at the end of it, not a design exercise. The pipeline around the encoder already exists, so a port supplies one thing: a module that turns a padded batch into per-token hidden states, plus a descriptor that says how those states become a vector. The BERT family page is the worked example of the result, and everything below is what it took to get there.

Read the repository first

Start with the file list, which costs nothing and tells you what the repository actually publishes:

hf download <org>/<name> --include "*.json" --include "1_Pooling/*" --dry-run

Seven files then decide the whole port.

New family checklist

1. Write candding/src/models/<family>/config.rs: a serde view of config.json with #[serde(default)] for every optional field and hidden_act typed as the shared Activation enum.
2. Build the architecture from the shared layers in embeddings.rs, attention.rs, layer.rs and model.rs. Never copy attention math into a family; call layers::attention::scaled_dot_product so the backend choice stays in one place.
3. Implement Encoder in model.rs: forward takes an EncoderInput and returns (batch, seq, hidden), alongside hidden_size and max_position_embeddings.
4. Add the model_type arm to models::load_encoder, which is what makes an unregistered repository of that architecture load without a registry entry.
5. Handle the weight prefix: try the bare tensor names first and fall back to the checkpoint prefix (bert., roberta., model.) with vb.contains_tensor.
6. Add the Family variant and the entries in candding/src/registry/<family>.rs, with pooling, templates, default length, license and phase.
7. Generate the references and wire up the tests: just fetch <id>, just golden <id>, then a golden_dense! line per model, and run the suite on CPU before touching an accelerator.
8. Write the family page under website/content/docs/models/<family>.mdx with its family and phase frontmatter and its pitfalls section, and run just docs-models so the catalog, the site table and the README table all come from the registry again.

Pitfalls

These are the mistakes that produce plausible but wrong vectors, in the order they tend to bite.

• GELU: a config that says gelu means the erf-based form; gelu_new and gelu_pytorch_tanh mean the tanh approximation. Both look fine and only one matches.
• LayerNorm epsilon differs by family, 1e-12 for BERT, 1e-5 for RoBERTa and ModernBERT, 1e-6 for the RMSNorm models. Read it from the config, never hardcode it.
• RoBERTa, XLM-RoBERTa and MPNet start position ids at padding_idx + 1, computed from the attention mask rather than from a plain range.
• BERT-style models add token_type_embeddings[0] to every token; skipping the lookup shifts every hidden state.
• With right padding and last-token pooling, take index length - 1 from the attention mask; the last column of the padded batch is not the last real token.
• The additive mask uses the dtype minimum rather than -inf, and it is added after the scores are scaled.
• The attention scale is 1 / sqrt(head_dim), applied to the query-key product before the mask.
• ALiBi, as JinaBERT uses it, has per-head slopes and adds its bias in every layer, with a maximum length of 8192.
• RoPE needs rope_theta, rope_scaling and any local and global theta pair from the config, applied to the query and key after the head split, in F32.
• Dense modules run after pooling and before normalization, in modules.json order, and their activation is usually Tanh or Identity.
• Matryoshka truncation comes after the Dense modules and before normalization, and the vector is re-normalized after truncating.
• Templates are part of the model, not of the call site: a missing query: prefix for an e5 model costs several retrieval points while still producing perfectly unit vectors.
• Tokenizer truncation counts special tokens, so the limit you set is the total length including them.
• Decoder-based embedders such as Qwen3 rely on the tokenizer's post-processor to append the end-of-sequence token; verify it with token_count on a short string before blaming the model.
• Mean pooling divides by the mask sum clamped at 1e-9, in the dtype of the hidden states.

Verify

just fetch <id>
just golden <id>
cargo test -p candding --features model-tests --test golden_<family> <module>
cargo test -p candding --features metal,model-tests --test parity
just docs-check

The golden run is the gate: cosine and max-abs against the reference for the raw, query and passage paths, plus the batching, norm, determinism, truncation and STS checks the testing page describes. Parity then confirms that the accelerator agrees with CPU on the same inputs, and just docs-check confirms that the site, the catalog data and the README table match the registry the port just changed. A model that fails parity on a backend ships with that backend marked as unverified in the catalog rather than with a skipped test.