Reasoning & Evaluation
23 min
Guarantees Without Calibration: Conformal Prediction and the Limits of LLM Confidence
A language model's stated confidence is a number, not a probability. Conformal prediction offers the opposite trade: it promises nothing about any single answer and something exact about the long run, from any scorer, with one as…