Probability Calibration: Platt Scaling and Isotonic Regression

A payments team reviews a transaction when expected fraud loss exceeds review cost. A review costs $20 and a fraud costs $500, so the rule is to review when \(p > 0.04\). The model has an AUC of 0.93, and it is useless for this rule, because among transactions it scores at 0.10 the observed fraud rate is 0.03. The ranking is excellent. The numbers are not probabilities. Any decision that multiplies a score by a cost, compares it to a fixed threshold, or sums scores into an expected count needs the second property, and AUC says nothing about it.

What calibration means and how it is measured

A binary classifier with score \(\hat p(x)\) is calibrated if

Discrimination and calibration are separate. AUC is invariant to any strictly increasing transform of the score, while calibration depends on the exact values, so a model can be excellent at one and poor at the other.

The standard diagnostic is a reliability diagram: bin predictions, and plot the observed positive rate in each bin against the mean prediction. Expected calibration error summarises it as \(\text{ECE} = \sum_b \frac{n_b}{n}\,|\bar y_b - \bar p_b|\), with \(n_b\) points in bin \(b\), observed rate \(\bar y_b\) and mean prediction \(\bar p_b\). The estimate depends on the binning, and Kumar, Liang and Ma proved that binning never overestimates, and can severely underestimate, the calibration error of a model with continuous outputs, so scaling methods such as Platt and temperature scaling are less calibrated than reported (Kumar, Liang & Ma, 2019, Verified Uncertainty Calibration, NeurIPS, arXiv:1909.10155). Proper scoring rules such as log loss and the Brier score reward calibration and discrimination together, and avoid the binning choice.

Different learners fail in characteristic directions. Across ten algorithms and eight problems, Niculescu-Mizil and Caruana found that max-margin methods, meaning SVMs and AdaBoost-style boosted trees and stumps, push predictions away from 0 and 1 and produce a sigmoid-shaped reliability curve, while naive Bayes pushes toward the extremes (Niculescu-Mizil & Caruana, 2005, Predicting Good Probabilities with Supervised Learning, ICML). Neural nets of that era and bagged trees were well calibrated out of the box. Random forests shrink away from the extremes for the vote-share reason described in bagging and random forests.

Platt scaling

Platt proposed fitting a sigmoid to the raw score \(f\):

with \(A\) and \(B\) chosen to minimise log loss on held-out data (Platt, 1999, Probabilistic Outputs for Support Vector Machines, in Advances in Large Margin Classifiers). To limit over-fitting, the 0/1 labels are replaced by smoothed targets \((N_+ + 1)/(N_+ + 2)\) and \(1/(N_- + 2)\), where \(N_+\) and \(N_-\) count positives and negatives. It is logistic regression with one feature, which gives two properties at once: it needs little data, and it can only correct distortions that a sigmoid can express. Being strictly monotone when \(A < 0\), it never changes the ranking. Temperature scaling, which divides multiclass logits by a single fitted constant, is its one-parameter multiclass descendant and was found surprisingly effective on modern deep networks (Guo et al., 2017, On Calibration of Modern Neural Networks, ICML, arXiv:1706.04599).

Isotonic regression

Isotonic calibration fits any non-decreasing step function \(m\) minimising \(\sum_i (y_i - m(f_i))^2\) (Zadrozny & Elkan, 2002, Transforming Classifier Scores into Accurate Multiclass Probability Estimates, KDD). The pool-adjacent-violators algorithm solves it in linear time after sorting: walk through labels in score order and, whenever a value falls below its predecessor, merge the two into their average and repeat backward.

With labels sorted by score as \(0, 1, 0, 0, 1, 1\), the second and third violate order and merge to \(0.5\); the fourth, \(0\), is below \(0.5\), so all three merge to \(1/3\). The fitted calibration map is \(0, \tfrac13, \tfrac13, \tfrac13, 1, 1\).

The flexibility is the trade-off. Niculescu-Mizil and Caruana varied calibration set size from 32 to 8,192: below roughly 200 to 1,000 examples Platt scaling beat isotonic regression for every learner, and with 1,000 or more examples isotonic was always as good or better. Isotonic also corrects shapes a sigmoid cannot, such as naive Bayes' inverted S.

When it breaks

Calibrating on training data calibrates nothing. A model that separates its training set perfectly gives Platt a 0/1 step to fit. The calibration set must be held out, or built from out-of-fold predictions.

Isotonic creates ties. A step function maps whole score ranges to one value, which can lower AUC and makes top-k selection arbitrary within a step. Some practitioners interpolate between steps; others accept the ties as honest resolution.

Calibration does not survive a change in base rate. A model calibrated at 1% prevalence is miscalibrated at 3%. Prior shift can be corrected analytically on the odds scale if the new base rate is known, but most deployments do not know it in real time.

Average calibration can hide subgroup miscalibration. A reliability curve on the diagonal overall can sit above it for one segment and below it for another. When decisions differ by segment, calibration has to be checked per segment, with each segment's sample size limiting what can be verified.

Language models add a separate problem. Calibration of token probabilities versus verbalised confidence behaves differently again, covered in calibration of language models.