Python

Install candding from PyPI, then embed and rerank from Python: the package wraps the Rust crate, keeps its method names, and returns NumPy arrays.

Install

pip install candding

Each wheel targets CPython's stable ABI, abi3, so one wheel per platform serves every Python version in its row. On any other platform pip builds the source distribution, which needs the Rust toolchain and C compiler that getting started lists; no wheel carries CUDA, so a CUDA build comes from the source distribution:

MATURIN_PEP517_ARGS="--features cuda" pip install candding --no-binary candding

Dense vectors

from candding import TextEmbedding

model = TextEmbedding("BAAI/bge-small-en-v1.5")
passages = model.passage_embed(["Embeddings map text to vectors.", "The cat sat on the mat."])
queries = model.query_embed(["what do embeddings do"])
print(passages.shape, passages.dtype)
print(queries @ passages.T)

passage_embed and query_embed apply the model's templates and embed applies none, as in Rust. Each row has unit length unless the model was built with normalize=False, so a dot product is a cosine similarity.

Sparse rows

from candding import SparseTextEmbedding

model = SparseTextEmbedding("prithivida/Splade_PP_en_v1")
passages = model.passage_embed(["Embeddings map text to vectors.", "The cat sat on the mat."])
query = model.query_embed(["what do embeddings do"])[0]
print(len(passages[0]), passages[0].indices.dtype, passages[0].values.dtype)
print([query.dot(passage) for passage in passages])

BM25

from candding import Bm25TextEmbedding

bm25 = Bm25TextEmbedding()
passages = bm25.passage_embed(["Embeddings map text to vectors.", "The cat sat on the mat."])
query = bm25.query_embed(["what do embeddings do"])[0]
print([query.dot(passage) for passage in passages])

Bm25TextEmbedding loads no weights and takes no device, dtype or batch size.

Multi-vector matrices

from candding import MultiVectorTextEmbedding

model = MultiVectorTextEmbedding("answerdotai/answerai-colbert-small-v1")
passages = model.passage_embed(["Embeddings map text to vectors.", "The cat sat on the mat."])
query = model.query_embed(["what do embeddings do"])[0]
print(query.shape, passages[0].shape)
print([model.score(query, passage) for passage in passages])

score is the late-interaction score under the checkpoint's own reduction, the score its reference ranks by; multi-vector explains the reductions.

Reranking

from candding import TextCrossEncoder

model = TextCrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
documents = ["The cat sat on the mat.", "Embeddings map text to vectors."]
print(model.rerank("what do embeddings do", documents))

rerank returns (document index, score) pairs best first; tied scores keep their input order and a NaN ranks last.

API

Every keyword after the model id is keyword-only.

Every model but Bm25TextEmbedding has the properties max_length, device, dtype, weights_dtype and descriptor, and TextEmbedding also has dim; Bm25TextEmbedding has descriptor. device and the dtypes are strings, and descriptor is a dict holding the fields candding describe --json prints. The model catalog lists every id.

Devices and dtypes

A device the build lacks raises CanddingError with the kind BackendNotCompiled rather than falling back to the CPU. Devices and dtypes has each dtype's limits.

Errors

The API page lists the variants.

Threads

Loading a model and every embed, count, score and rerank call release the GIL while the Rust code runs, so Python threads can call one model at once. On the CPU their forward passes run in parallel; on Metal the crate serializes every forward pass in the process behind one lock.

The Rust names

embed is the passage side on SparseTextEmbedding, Bm25TextEmbedding and MultiVectorTextEmbedding, and applies no template on TextEmbedding, as in Rust. The API page has the Rust types.

Verify

The dense example prints the array's shape and dtype, then one row of scores in which the first passage scores higher; the last digits can differ between machines:

(2, 384) float32
[[0.7862278  0.35534123]]

The pytest suite in candding-py/tests checks that every Python output equals the Rust crate's bit for bit on the CPU, and that the dense vectors match the reference implementation's within the F32 tolerance.