Concepts

An embedding is only useful when it matches the vectors the model's authors published, and matching them is a matter of getting a short list of details right: which template the text goes through, how the tokenizer pads and truncates, which token or which average becomes the sentence vector, and whether the result is normalized. candding keeps each of those details in one module, so the same pipeline runs every family and every backend.

Nothing on this list is guessed at load time. A ModelDescriptor carries the dimension, the default maximum length, the pooling mode, the normalization flag, the query and passage templates, the default instruction, the Dense module directories and the Matryoshka flag; the registry ships one descriptor per catalog model, and a repository outside the registry either gets a descriptor from you or one read from its own sentence-transformers files. Family code never sees the descriptor: it turns a padded batch into per-token hidden states and stops there.

Device and dtype choices sit outside the pipeline as well. Model code has no backend branches, so switching from CPU to Metal changes which attention kernel runs and nothing else, and every pooled vector is upcast to F32 before it is truncated, normalized and handed back as Vec<f32>.

• Pipeline: every stage from text to vector, the tensor shapes between them, length-sorted batching and the effective maximum length.
• Pooling and templates: the three pooling modes, which catalog model uses which, the query and passage templates, instructions and Matryoshka truncation.
• Devices and dtypes: device selection order, F32 as the default dtype, the single attention branch and the feature flags.
• Model files and cache: the Hugging Face cache layout, which files are downloaded and read, gated repositories and local directories.