API
The public types of the candding crate, the feature flags, the compatibility guarantees and where the generated reference lives.
The public surface is one model type and one builder per output, the descriptors, the catalog lookups in registry, the multi-vector scoring functions, one error enum and the device helpers; everything else is #[doc(hidden)] and outside semver.
Models
| Type | Output | Methods |
|---|---|---|
TextEmbedding | one Vec<f32> per text | query_embed, passage_embed and embed, each with a _dim form for Matryoshka truncation; query_embed_with_instruction(texts, instruction, batch_size); token_count without a template, and query_token_count, passage_token_count and query_token_count_with_instruction(texts, instruction) with one; dim |
SparseTextEmbedding | one SparseEmbedding per text | query_embed, passage_embed, embed (the passage side) and token_count |
Bm25TextEmbedding | one SparseEmbedding per text, with no Result | query_embed, passage_embed and embed (the passage side), with no device and no dtype |
MultiVectorTextEmbedding | one MultiVectorEmbedding per text | query_embed, passage_embed, embed (the passage side) and token_count |
TextCrossEncoder | one RerankScore per document, in input order, never sorted | rerank(query, &documents, batch_size) and rerank_with_instruction; pair_token_count and pair_token_count_with_instruction, summed over the windows for a windowed entry; pair_window_count, 1 for a pair or prompt entry |
Every type has descriptor, and all but Bm25TextEmbedding have max_length, device, dtype and weights_dtype, the dtype the loaded weights are in. Each method's batch_size: Option<usize> overrides the builder's for that call.
Builders
Each builder comes from its type's builder(id) and has device, dtype, max_length, batch_size, revision and descriptor. The id can be a local directory, which the sparse, multi-vector and rerank builders load only with a descriptor; getting started has the defaults.
| Builder | Batch size | Also has | Refuses |
|---|---|---|---|
TextEmbeddingBuilder | 32 | normalize, whose false returns the raw pooled vector | an id only another catalog carries, with NonDenseModel |
SparseTextEmbeddingBuilder | 8 | — | an id outside its catalog without a descriptor, with UnregisteredModel; Qdrant/bm25, with ComputedSparseModel |
MultiVectorTextEmbeddingBuilder | 8 | normalize, whose false returns each row as the projection writes it | an id outside its catalog without a descriptor, with UnregisteredModel |
TextCrossEncoderBuilder | 8 | — | the same |
Bm25TextEmbeddingBuilder comes from Bm25TextEmbedding::builder(), takes no parameters, and its build() returns the model, not a Result.
Outputs and scores
| Type | What it is |
|---|---|
SparseEmbedding | one sparse row, indices and values ascending by index; from_pairs keeps the larger weight of a repeated index and drops zeros, and dot scores two rows |
MultiVectorEmbedding | one per-token matrix in a row-major buffer: from_rows, dim, row_count, is_empty and rows |
multi_vector::late_interaction_score | MaxSim of a query against a document, with an optional mask of each side's real rows |
multi_vector::late_interaction_score_for | the same under a checkpoint's ScoreReduction and MaskedRowFill, with no masks |
ScoreReduction | Sum or MeanByQueryLength |
MaskedRowFill | DtypeMinimum or Zero, the value a masked document row takes before the maximum |
RerankScore | one pair's score: value() is the f32, and Ord sorts every NaN below every number and numbers as f32::total_cmp does, so a sort never panics |
ScoreActivation | Identity, the head's raw output, or Sigmoid, a probability |
Descriptors and the registry
| Type | What it is |
|---|---|
ModelDescriptor | a dense model's facts beyond its weights, from pooling and templates to the default dtype and statuses; new(id, family, dim, pooling) |
SparseDescriptor | a sparse model's, with its head, vocabulary size and any second repository; new(id, family, head, vocab_size) |
MultiVectorDescriptor | a multi-vector model's, with its head, projection, markers, query length and scoring conventions; new(id, family, head, projection_in, projection_out) |
RerankDescriptor | a reranker's, with its head, score activation, input and position conventions; new(id, family, head, score_activation, input) |
SparseHead | SpladeMlm, M3Linear, SpladeTied, MiniCoil, Bm42 or Bm25 |
MultiVectorHead | M3Colbert, StanfordColbert or JinaColbert |
RerankHead | BertPooler, RobertaClassificationHead or LogitScore { true_token_id, false_token_id, unembedding }, the first id's logit minus the second's |
Unembedding | Tied or Untied: a LogitScore head reads the token embedding table or lm_head.weight |
RerankInput | Pair { truncation }, Prompt { template, default_instruction } or Windowed { query_cap, overlap }; check_instruction(model, instruction) refuses an instruction for a pair or windowed input |
PairTruncationPolicy | LongestFirst or PreserveQueryToThreeQuarters |
PromptTemplate | Qwen3Reranker, with prefix(), suffix(), content(..) and render(..) |
PositionScheme | Rotary or Absolute, how a Family::XlmRobertaFlash checkpoint numbers its tokens |
AlibiSlopeVariant | Canonical or HalvedInterpolatedTail, a Family::JinaBert checkpoint's ALiBi slopes; None reads as Canonical |
SparseComputation | Encoded, a forward pass, or Computed, no encoder, as for Qdrant/bm25 |
WeightsFormat | Safetensors, Pickle, Npy or NoWeights, which decides the files a download fetches |
Family, Pooling | the architecture, with the mapping from a model_type string; CLS, mean or last token |
BackendSupport, DtypeBackendSupport | a model's CPU, Metal and CUDA status, and each dtype's through row(dtype) and status(dtype, device); untested() builds either |
Support | Verified, Pending, Untested, Unsupported or NotApplicable, for an entry with no device or dtype axis |
registry::dense, registry::sparse, registry::multi_vector, registry::rerank | each catalog's all, lookup (case-insensitive), by_phase and dtype_needs_accelerator; registry::sparse also has head_source_format and head_source_revision |
registry::weights_format, registry::canonical_id | across every catalog: an id's weight format, safetensors when no catalog carries it, and the catalog's spelling of an id |
Errors
CanddingError is every failure the crate produces, and Result<T> its alias; the variants a caller branches on carry typed fields:
| When | Variants |
|---|---|
a download fails, each with the model's id | GatedModel, RepositoryNotFound, RevisionNotFound { revision }, Network, DownloadFailed, UnsafeHubPath { path, reason }, and Offline under HF_HUB_OFFLINE |
the cache lacks a model and the build has no download feature | ModelNotFound { id, hint }, the hint being the candding fetch command |
| a device cannot be created | BackendNotCompiled { backend } when the build leaves out its feature, Device { backend, reason } otherwise |
| a dtype is refused | UnsupportedOnDevice { backend, dtype } on one device, UnsupportedDtype { dtype } on every device |
| the weights exceed the device's memory budget | InsufficientMemory { backend, needed_bytes, available_bytes }, refused before any weight is read |
| the id belongs to another builder | NonDenseModel { catalog }, UnregisteredModel { catalog }, ComputedSparseModel |
| a length or a dimension is out of range | LengthTooSmall { minimum, kind }, whose LengthKind is Text, Pair, Prompt, Windowed or QueryCap; MrlUnsupported; InvalidDim |
| an instruction reaches a pair or windowed reranker | InstructionOnPairInput |
| the checkpoint contradicts its descriptor | UnsupportedArchitecture, DenseMismatch, PromptSuffixMismatch, UnembeddingDisagreesWithConfig, TiedEmbeddingsWithLmHead, MissingLmHead, PositionSchemeDisagreesWithConfig, LogitTokenOutOfRange, ProjectionInMismatch, MissingQueryLength, MissingHeadSource; Tokenizer for a windowed entry whose pair template lacks two separators |
Backend is Cpu, Metal or Cuda, and Catalog is Sparse, Bm25, MultiVector or Rerank, the catalog beside the dense one that an error names.
Devices
| Item | What it is |
|---|---|
Device, DType | candle's types, re-exported so callers need no direct candle dependency |
device | auto, cpu, metal(ordinal), cuda(ordinal) and describe; devices and dtypes has what each returns |
Feature flags
| Feature | Default | What it does |
|---|---|---|
download | on | downloads a missing model from the Hugging Face Hub in Rust; without it the builders read only the cache and local directories |
model-tests | off | test-only, enables the suites that need downloaded weights |
metal, cuda, cudnn, accelerate, mkl | off | the backends, on devices and dtypes |
Compatibility
| Property | Value |
|---|---|
| Edition | 2024 |
| Minimum Rust | 1.94 |
| candle | 0.11, for candle-core and candle-nn |
| Runtime dependencies | no ONNX runtime, no Python and no candle-transformers; the download feature compiles aws-lc-sys from C, as getting started lists |
| License | Apache-2.0 OR MIT, in LICENSE-APACHE and LICENSE-MIT; NOTICE records the model implementations derived from Hugging Face text-embeddings-inference |
| Model weights | each model's own licence, in the catalog |
| Changes | CHANGELOG.md |
Stability
| Rule | What it means |
|---|---|
| Versions | in 0.x, a breaking change bumps the minor version |
| Hidden items | anything #[doc(hidden)], the implementation modules included, is outside semver |
| Enums | #[non_exhaustive], so a match needs a wildcard arm, except Unembedding, SparseComputation and MaskedRowFill, which have two values by definition |
| Descriptors | the four descriptors, BackendSupport and DtypeBackendSupport are #[non_exhaustive] with public fields: build one with its constructor, then assign fields |
| Catalog | new entries and status changes can ship in any release |
Reference documentation
The item-level reference is on docs.rs from the first crates.io release; until then, generate it locally:
cargo doc -p candding --no-deps --open