Learned Sparse Retrieval

Dense retrieval fixed vocabulary mismatch and gave up exact matching, interpretability and the inverted index. Learned sparse retrieval asks whether the first can be had without the rest: keep the sparse, term-indexed representation, and let a transformer decide the weights.

The output for a document is a vector over the model's vocabulary, mostly zeros, where a non-zero entry means "this term is relevant to this document" whether or not it occurs in the text. A document about cars can carry weight on "automobile" and "vehicle". Scoring is a dot product between query and document vectors, which over a sparse vocabulary basis is exactly what an inverted index computes.

How the weights are produced

SPLADE derives the representation from the masked language modelling head (Formal, Piwowarski and Clinchant, 2021, SPLADE, SIGIR, arXiv:2107.05720). For each input token, the MLM head produces logits over the whole vocabulary; these are aggregated across positions with a log-saturation transform and a max or sum pooling, giving one weight per vocabulary term:

The log-saturation is not incidental. It plays the role BM25's \(k_1\) term plays, preventing any single term from dominating, and it makes the resulting weights behave enough like term frequencies for an inverted index's assumptions to hold approximately.

Sparsity is enforced by an explicit regulariser, typically FLOPS, which penalises the expected cost of the dot product rather than the count of non-zeros directly. This is the design decision that distinguishes the approach: sparsity is a trained objective with a tunable weight, so the accuracy-latency trade is a knob rather than a consequence.

What it buys and what it costs

The appealing properties are real. Exact term matching survives, so identifiers, product codes and names still work, which is where dense retrieval is weakest. The representation is inspectable: you can read which terms a document was expanded with and why a match happened, which no dense embedding supports. And it runs on the same inverted index infrastructure, with the same compression, sharding and update machinery.

The costs are equally real and less advertised. Expanded documents have far more non-zero terms than the original text, so postings lists lengthen and the index grows. Query vectors have many non-zeros with weights unlike a keyword query's, which is exactly the condition under which dynamic pruning is least effective, so the per-query cost is materially higher than BM25 on the same corpus. The efficiency claim rests on "it is an inverted index", and the constant factors differ substantially.

Document-side-only expansion, where the query stays a plain bag of words and only documents are expanded, is the standard compromise: it preserves most of the effectiveness gain and keeps query processing close to BM25's cost.

When it breaks

Domain transfer is not automatic. The weights are learned from a training distribution, typically MS MARCO, and carry its notion of relevance. On a corpus with different vocabulary and different query intent, a learned sparse model can underperform BM25, which has no parameters to transfer badly.

Vocabulary is fixed to the model's tokeniser. Terms outside it are expressed as subwords, so rare identifiers and domain jargon are represented by pieces whose learned weights are unreliable. This partly erodes the exact-matching advantage that motivated the approach.

Indexing cost is a forward pass per document. Unlike BM25, which needs only tokenisation, every document must go through the model, and re-indexing after a model update means reprocessing the whole corpus. That is the same operational burden dense retrieval carries.

It is a first-stage ranker. Like BM25 and dense retrieval, it is designed to produce candidates. A cross-encoder reranker over the top few hundred remains the largest single accuracy lever in most pipelines, and comparisons that omit the reranker from both sides can overstate the importance of the first stage.

Comparisons need matched efficiency budgets. Reporting effectiveness without latency and index size is the recurring problem in this literature. A model that beats BM25 by three nDCG points at four times the query cost is a different proposition from one that does it at parity, and only the first number usually appears in the abstract.