Choosing and Adapting an Embedding Model

The embedding model is the one component of a RAG stack that is hard to change later. Swap the reranker and you re-run a scoring pass; swap the LLM and you re-run evaluations; swap the embedding model and you re-embed the entire corpus and rebuild every index. It is also the component most often selected by reading the top row of a leaderboard, which is where the trouble starts.

What MTEB does and does not tell you

The Massive Text Embedding Benchmark covers 8 task types across 58 datasets and 112 languages, and its founding result is the useful one: across 33 models evaluated, no method dominated all tasks (Muennighoff et al., 2023, MTEB: Massive Text Embedding Benchmark, arXiv:2210.07316). The benchmark's own conclusion is that there is no universal text embedding.

Two practical consequences follow. First, the aggregate average is a poor selector, because it mixes classification, clustering, reranking, and retrieval, and you probably care about exactly one of them; read the retrieval sub-scores, and preferably the sub-scores on the dataset closest to your domain. Second, leaderboard saturation invites overfitting. A model tuned to score on public benchmarks has seen those datasets discussed, if not trained on, and the gap between benchmark rank and your-corpus rank is routinely larger than the gap between adjacent leaderboard positions. Build a few hundred labelled query-document pairs from your own traffic and evaluate on those; it will tell you more than the whole leaderboard.

The properties that actually constrain you

Dimension sets your memory and latency budget. An HNSW index over 10 million chunks at 1536 dimensions in float32 is roughly 61 GB of raw vectors before graph overhead; at 384 dimensions it is about 15 GB (see ANN indexes: HNSW, IVF, PQ). Matryoshka representation learning changes the shape of this decision by training a single model whose prefixes are themselves valid embeddings, so one 1536-dimension vector can be truncated to 256 dimensions for a cheap first-stage search and used in full for rescoring, with no second model and no extra inference cost (Kusupati et al., 2022, Matryoshka Representation Learning, arXiv:2205.13147). If a model advertises Matryoshka training, dimension stops being a one-way door.

Sequence length determines how much of a chunk actually reaches the encoder. A model with a 512-token limit silently truncates your 800-token chunks, and the tail of every chunk is unindexed. This mismatch is common and produces retrieval failures that look like model quality problems.

Symmetry decides whether queries and documents need different treatment. Models trained with instruction prefixes or with separate query and passage encoders expect you to use them that way; embedding a short question with the passage prefix quietly costs several points of recall. Read the model card and follow it exactly.

Adapting rather than replacing

Fine-tuning an embedding model on in-domain pairs is cheaper than most teams assume and usually beats moving one or two leaderboard positions. The lever that matters is hard negatives: training against random negatives teaches the model to separate a legal contract from a cooking recipe, which it already does, while training against near-miss passages retrieved by the current system teaches it the distinction your users actually trip on. The standard recipe is to mine top-ranked-but-wrong passages from production logs, verify a sample by hand, and train with a contrastive objective on those.

An adapter is the lighter-weight version: freeze the encoder and learn a small linear or low-rank transform on top of its output. Because the base vectors are unchanged, you can re-project an existing index without re-running the encoder over the corpus, which turns a full re-embedding into a matrix multiply.

When it breaks

• Re-embedding is a migration, not a config change. Two embedding spaces are not comparable, so the switchover needs a dual-index period, a backfill, and a cutover. Budget for it before you commit to a model.
• Fine-tuning overfits to logged traffic. Hard negatives mined from today's system encode today's distribution; the model gets better at queries you already see and no better at the long tail. Hold out a time-shifted slice to detect this.
• Domain gaps are worse than they look. General-purpose encoders handle code, legal citations, chemical names, and product SKUs poorly, and the failure is silent because every vector still has a plausible nearest neighbour.
• Multilingual coverage is uneven within a single model. A model covering 100 languages does not cover them equally; check the specific languages you serve rather than the headline count.
• Cosine similarity scores are not calibrated across models or corpora. A 0.82 in one system means nothing in another, so any absolute similarity threshold you tuned is invalidated the moment the model changes. Prefer rank-based cutoffs and a reranker with a calibrated score.