Generative Retrieval for Recommendation

Retrieval has had the same shape for a decade. Embed the user, embed every item, and find the nearest neighbours with an approximate index. The index is a separate system with its own memory footprint, its own build pipeline and its own failure modes, and it grows with the catalogue. Two-tower retrieval is the canonical form.

Generative retrieval deletes the index. A sequence model reads the user's history and emits the identifier of the next item, one token at a time, the way a language model emits a word. The idea came from document search, where the differentiable search index showed that a single transformer could map queries directly to document identifiers with all corpus information held in its parameters (Tay et al., 2022, Transformer Memory as a Differentiable Search Index, NeurIPS 2022). TIGER carried it into recommendation by making the identifier a semantic ID rather than an arbitrary number (Rajput et al., 2023, NeurIPS 2023).

Decoding an address instead of ranking a catalogue

Training is ordinary sequence-to-sequence. The input is the user's interaction history rendered as a flat token stream of semantic IDs; the target is the semantic ID of the held-out next item. At serving time you run beam search over the code tree: the first decoding step chooses among 256 first-level codes, the second among the 256 children of the chosen prefix, and so on. A beam of width \(B\) over \(L\) levels visits \(O(B \cdot L \cdot K)\) candidate continuations, and \(K\) and \(L\) are small constants, so the cost does not depend on catalogue size at all.

That is the structural win, and it is real. Retrieval cost becomes a function of identifier length rather than corpus size, and the memory that an ANN index would have consumed is replaced by decoder parameters that were going to exist anyway.

The decode must be constrained. An unconstrained decoder will happily emit a code tuple that addresses no item, so production implementations mask the logits at each step to the children that actually exist in a prefix trie of the live catalogue. Without that mask, invalid identifiers appear in the beam and the effective recall drops by whatever fraction of the beam they occupy.

On the standard Amazon Reviews benchmarks TIGER reports Recall@5 of 0.0441 and NDCG@5 of 0.0309 on Beauty, with gains of up to 29 percent in NDCG@5 over SASRec and 17.3 percent in Recall@5 over S³-Rec.

The failure the benchmarks average away

Aggregate recall hides where generative retrieval loses, and a controlled comparison against sequential dense retrieval found the gap concentrated almost entirely in one place: on cold-start items, generative retrieval achieves near-zero performance (Yang et al., 2024, Unifying Generative and Dense Retrieval for Sequential Recommendation, arXiv:2411.18814). The mechanism is specific. The decoder's conditional probabilities are fitted to identifiers it saw during training, and for a genuinely new item the minimum generation probability anywhere in the beam exceeds the probability the model assigns to the correct new identifier. The item is not ranked low; it is unreachable.

This is the exact opposite of the pitch. Semantic IDs generalise to unseen items as features, because the codes are shared. A decoder trained on observed identifier sequences does not inherit that generalisation, because what it learned is a distribution over paths it has walked.

LIGER's answer is to stop treating this as a choice: generate a small candidate set, explicitly inject cold-start items, then re-rank the union with a dense scorer. That keeps the small storage footprint and recovers the cold-start recall, at the cost of running both systems.

When it breaks

Tree structure correlates the scores of unrelated items. Items sharing a prefix share every factor in the probability chain up to the point they diverge, so the model assigns them similar probabilities regardless of the user. There are simple preference patterns that ordinary collaborative filtering represents easily and an autoregressive semantic ID decoder provably cannot, and injecting a latent token before each identifier to split the single tree into several recovers an average of 3.45 percent NDCG@10 (2026, Expressiveness Limits of Autoregressive Semantic ID Generation, arXiv:2605.06331).

Beam search is serial where scoring is parallel. An ANN lookup is one round trip. A four-level decode is four dependent forward passes inside a retrieval budget usually measured in tens of milliseconds, and you cannot batch away a data dependency.

You cannot add an item without touching the model. An ANN index accepts an insert. A decoder has to learn the new path, which is the catalogue update problem.

Offline gains on academic splits are weak evidence. This literature evaluates on the same Amazon and MovieLens splits whose protocol problems are well documented; see the progress illusion in recommender systems before promoting a reported gain to a roadmap item.