Neural Text Degeneration

"The model sounds robotic" and "the model keeps repeating itself" are usually not model problems at all. They are decoding problems, specific, well-studied, and named. Holtzman et al., 2019, in "The Curious Case of Neural Text Degeneration," gave the phenomenon its name and its first rigorous diagnosis: text generated by maximisation-based decoders, greedy-decoding and beam-search, is measurably, quantifiably worse than either human-written text or text sampled from the model's own distribution, despite having higher probability under the model. This concept is the underlying diagnosis that nucleus-sampling-top-p, min-p-and-typical-sampling, and repetition-penalty were each built to treat.

The paradox, stated precisely

A well-trained language model assigns high probability to fluent, plausible text; that is what training on next-token prediction optimises for (see next-token-prediction-cross-entropy). It is natural to assume, then, that the highest-probability continuation is also the best continuation, and that a decoder which searches harder for high-probability sequences (beam search over greedy) should produce better text. Holtzman et al. show this intuition is wrong for open-ended generation. As beam width increases, and as decoding leans harder into maximisation, output quality by human judgement does not improve monotonically, it degrades: text becomes more repetitive, more generic, and measurably less like human text on distributional statistics, even as its probability under the model climbs.

The evidence: three distinct symptoms

The paper documents this with several concrete measurements, not just a qualitative impression.

Repetition. Maximisation decoders fall into self-reinforcing loops: once a phrase is produced, the context that contains it makes reproducing that phrase the (locally) highest-probability continuation again, with no mechanism to break out (see greedy-decoding for the mechanics of why). The paper shows this quantitatively via repeated n-gram counts, which are dramatically elevated in beam-search output relative to human text.

Probability concentration mismatch. The paper compares the per-token probability the model assigns to its own generated tokens against the probability it assigns to the actual next token in human-written text. Human text is not simply "the most likely word every time"; the per-token probability trace of real text zigzags, is sometimes highly predictable, sometimes genuinely surprising. Maximisation decoding produces a probability trace that stays implausibly high and flat throughout, which is itself a statistical fingerprint distinguishing machine text from human text, and a giveaway that the generation process, not the model's underlying knowledge, is the source of the unnaturalness.

Boring, generic content. Independent of repetition, maximisation-decoded text skews toward safe, high-frequency, low-information phrasing, because generic continuations are, definitionally, the ones many training documents agree on and therefore assign consistently higher probability to. Sampling from the actual distribution, properly truncated (see nucleus-sampling-top-p), recovers some of the specificity and variation that maximisation systematically discards.

Why sampling helps, and why unrestricted sampling doesn't

The naive fix, sample directly from the model's untruncated distribution instead of maximising, trades one failure mode for another. Ancestral sampling at T=1 with no truncation samples from the full long tail of the vocabulary, including tokens the model assigns only a sliver of probability to precisely because they are genuinely poor continuations. Over a long enough generation, sampling from that tail even occasionally is enough to derail the sequence into incoherence, since each token conditions on everything before it, and one bad token corrupts the context for every subsequent step.

Nucleus sampling's core insight is that neither extreme is right: the fix is not "maximise harder" or "sample everything," it is "sample, but only from the part of the distribution the model actually stands behind." That's the entire argument for truncate-then-sample as a decoding family, and it explains why virtually every production decoder for open-ended text (top-k, top-p, min-p, all discussed elsewhere in this bundle) shares that same two-step shape: narrow the candidate set, then sample within it.

A different fix: contrastive search

Not every response to degeneration is truncate-then-sample. Su et al., 2022 propose contrastive search, which stays fully deterministic (no sampling at all) but explicitly penalises candidates that are too similar, in representation space, to what has already been generated, directly targeting the repetition symptom without relying on randomness to avoid it. It is evidence that "degeneration" and "determinism" are not the same axis: you can fix repetition without introducing randomness, and you can introduce randomness without fixing repetition, they are separate problems that happen to often be discussed together.

When it falls down

• The diagnosis is strongest for open-ended generation. For tasks with a genuinely narrow correct-answer space (translation, extraction, classification), higher sequence probability really does correlate with correctness, and the degeneration argument does not transfer; this is why beam search remains standard in machine translation.
• Sampling trades one failure mode for another if untruncated. The paper's own results are part of the argument for truncation, not against maximisation-avoidance generally; "just sample" without a truncation step reintroduces incoherence.
• The statistical fingerprints are properties of decoding, not detection guarantees. The probability-trace flatness the paper documents has been used as a signal in machine-text detection research, but detection is a moving target as decoders and models change; it should not be treated as a settled, robust classifier.
• Newer instruction-tuned and RLHF'd models shift the picture somewhat. Models tuned with human feedback often have sharper, more calibrated distributions than the base models the original paper studied, which changes (without eliminating) how visible these degeneration symptoms are in default sampling settings.

Further reading

• Holtzman et al., 2019, The Curious Case of Neural Text Degeneration, arXiv:1904.09751 - the paper that named and diagnosed the phenomenon, and introduced nucleus sampling as a response.
• Su et al., 2022, A Contrastive Framework for Neural Text Generation, arXiv:2202.06417 - contrastive search, a deterministic alternative response to the same diagnosis.
• Zhang, Duckworth, Ippolito, Neelakantan, 2020, Trading Off Diversity and Quality in Natural Language Generation, arXiv:2004.10450 - a systematic empirical study of the diversity/quality tradeoff across decoding strategies.