Epistemic and Aleatoric Uncertainty in LLMs

Two failures produce the same low-confidence signal. Ask "What is the capital of a country I just invented?" and the model does not know, which more data or a better model could fix; that is epistemic uncertainty. Ask "Write a sentence about the sea" and there are millions of good answers, which nothing fixes and nothing should; that is aleatoric uncertainty, irreducible by construction. Any monitoring system that treats a single entropy number as "confidence" is summing these two and acting on the total.

Token entropy measures the wrong thing

The obvious signal is the entropy of the model's output distribution, or the length-normalised log-probability of the generated sequence. Both are dominated by lexical freedom. "Paris", "It is Paris", and "The capital is Paris" are three distinct sequences with three different probabilities, so a question the model answers correctly and consistently can register high entropy purely because English offers many ways to say the same thing.

Semantic entropy repairs this by clustering samples before measuring. Draw multiple generations, group them into meaning classes using bidirectional entailment (each sample must entail the other for them to share a cluster), sum probability mass within each cluster, and compute entropy over clusters rather than sequences (Kuhn et al., 2023, Semantic Uncertainty, arXiv:2302.09664). Low semantic entropy means the model keeps saying the same thing in different words; high semantic entropy means it keeps saying different things. The extension of this method to detecting confabulations, that is, fluent claims generated arbitrarily rather than from knowledge, was published in Nature (Farquhar, Kossen, Kuhn and Gal, 2024, Detecting hallucinations in large language models using semantic entropy, Nature 630(8017), 625–630, doi:10.1038/s41586-024-07421-0).

The other estimators, and what they cost

Deep ensembles. Train several models with different seeds; disagreement between them is a well-behaved epistemic signal and the strongest classical baseline (Lakshminarayanan et al., 2016, arXiv:1612.01474). At LLM scale the cost is prohibitive, though ensembles over fine-tuned adapters or over prompts recover part of the effect cheaply.

MC dropout. Keep dropout on at inference and sample several forward passes, interpretable as approximate Bayesian inference (Gal and Ghahramani, 2015, arXiv:1506.02142). Modern LLMs largely do not train with dropout, so this is mostly unavailable.

Self-evaluation. Ask the model to score the probability that its own answer is correct. Large models are reasonably calibrated on multiple-choice and true/false formats when the format is right, and the P(True) approach extends this to open-ended generation by having the model propose answers and then evaluate them (Kadavath et al., 2022, Language Models (Mostly) Know What They Know, arXiv:2207.05221). The catch is that RLHF degrades the calibration that base models have, so a well-calibrated base checkpoint does not imply a well-calibrated chat model.

Using the distinction

The two kinds of uncertainty call for opposite responses. High epistemic uncertainty is a retrieval problem: fetch documents, call a tool, or escalate to a human, because the information is missing. High aleatoric uncertainty is a specification problem: the question admits many answers, so either accept any of them or constrain the request. Routing on total uncertainty sends both to the same fallback, which means burning retrieval budget on questions that were merely open-ended.

When it breaks

• Sampling cost. Semantic entropy needs 5 to 10 generations per query plus entailment checks. That is an order of magnitude more expensive than the answer itself, which restricts it to high-stakes paths or to offline monitoring.
• The entailment judge is a model too. Cluster quality depends on an NLI model or an LLM judge, and its errors propagate directly into the uncertainty estimate.
• Confidently wrong is a real state. A model that has memorised an incorrect association is consistent across samples, so semantic entropy is low and the answer is still wrong. Consistency measures disagreement, not truth.
• Calibration drifts with every change. Thresholds tuned on one model version, prompt, and temperature do not survive any of those changing. Recalibrate as a release step.