Min-p and Typical Sampling

Nucleus sampling improved on top-k by truncating on cumulative probability instead of a fixed count (see nucleus-sampling-top-p), but cumulative probability is itself only one way to define "the reasonable part of the distribution." Two later methods, min-p and locally typical sampling, each argue that top-p's own definition of "reasonable" is subtly wrong in a specific, fixable way.

Min-p: scale the floor to the top token

Top-p's failure mode appears at high temperature. Temperature flattens a distribution before top-p ever sees it (implementation order varies, but conceptually), and a flattened distribution can push the cumulative-probability threshold p deep enough into the tail to admit tokens that would have been obviously implausible at the model's original, unflattened confidence. The nucleus grows not because the model became more genuinely uncertain, but because temperature mechanically stretched the gaps.

Min-p sidesteps this by defining the candidate threshold relative to the most probable token rather than as an absolute cumulative sum. A token is admitted if its probability is at least min_p times the top token's probability:

threshold = min_p * p_max
keep token i  if  p_i >= threshold

If the model is very confident (p_max close to 1), the threshold is high in absolute terms, so only genuinely competitive tokens survive, min-p stays tight exactly when top-p's temperature-inflated nucleus would loosen. If the model is uncertain (p_max modest), the threshold scales down proportionally, allowing a wider, appropriately diverse set. Because the threshold is always defined relative to p_max, min-p remains well-behaved across a much wider range of temperatures than top-p does, which is its practical selling point: it lets you push temperature higher for genuine creative diversity without the output collapsing into the incoherence that high-temperature top-p is prone to. This method (min-p sampling) was proposed by Nguyen et al., 2024, who report it holding up better than top-p specifically in the high-temperature regime.

Locally typical sampling: truncate by information content, not rank

Typical sampling, from Meister et al., 2022, starts from a different premise entirely. Both top-k and top-p implicitly assume "most probable" is the right criterion for "reasonable to generate." Typical sampling argues this is not what human language does: human text is not composed of a sequence of maximally-probable-given-context words. It contains a mix of expected and mildly surprising words, and the amount of information (measured in bits, via entropy) each word carries tends to sit near the local expected value, neither maximally predictable nor maximally surprising.

Formally, typical sampling keeps tokens whose information content, -log p(token), is close to the conditional entropy of the distribution at that step, H = -sum_i p_i * log p_i, rather than keeping tokens purely by probability rank:

keep token i  if  | -log(p_i) - H |  is small

This can, deliberately, exclude the single most probable token if that token is so predictable that including it would push the sample toward the same dull high-likelihood text that plagues greedy and beam decoding (see neural-text-degeneration). It targets a different failure mode than top-p: not "the tail is too long" but "always picking the most expected word is itself unnatural."

How the two relate

Min-p and typical sampling both start from the observation that top-p's cumulative-probability criterion is a reasonable but not uniquely correct definition of "the good part of the distribution." Min-p keeps probability as the currency but changes the reference point (relative to the top token instead of an absolute cumulative sum). Typical sampling changes the currency entirely (information content instead of raw probability). In practice they are used for different goals: min-p for robustness at high temperature and general-purpose chat, typical sampling more often discussed in research contexts probing what "human-like" text statistics actually look like.

When it falls down

• Min-p's ideal range is narrow and model-dependent. min_p values are typically small (roughly 0.02 to 0.1), and the right value shifts with model calibration; poorly calibrated models (very sharp or very flat native distributions) need different settings than the papers' benchmarks used.
• Typical sampling can reject the actually-correct token. By design it may exclude the single most probable continuation if that continuation carries "too little" information relative to the local entropy, which is a liability on tasks (factual QA, code) where the most probable token usually is the right one.
• Adoption is uneven. Not every inference API exposes min_p or a typical-sampling parameter; where the parameter is missing, top-p plus a moderate temperature remains the fallback, and the specific robustness gains reported in the papers are not directly available.
• Both add a hyperparameter to an already crowded space. Stacking temperature, top-k, top-p, min-p, and repetition penalties in one request multiplies the tuning surface; interactions between them are not always documented precisely by a given serving stack.

Further reading

• Nguyen et al., 2024, Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs, arXiv:2407.01082 - introduces min-p and its high-temperature robustness experiments.
• Meister et al., 2022, Locally Typical Sampling, arXiv:2202.00666 - introduces typical sampling via the information-content argument.
• Holtzman et al., 2019, The Curious Case of Neural Text Degeneration, arXiv:1904.09751 - the nucleus sampling paper both methods position themselves against.