Missing Data Mechanisms

Mean-imputing a column that is 30% missing is a decision with statistical consequences, and which consequences depends entirely on a question the data cannot answer: why is it missing? Rubin's taxonomy is the standard framing and it is worth stating precisely, because the three cases license completely different methods (Rubin, 1976, Inference and Missing Data, Biometrika 63(3)).

MCAR, missing completely at random: missingness is independent of everything, observed and unobserved. A sensor dropped packets at random. Complete-case analysis is unbiased here and merely wasteful.

MAR, missing at random: missingness depends on observed variables but not, given those, on the missing value itself. Income is missing more often for younger respondents, and age is recorded. Conditioning on the observed variables makes the missingness ignorable, which is the assumption under which principled imputation is valid.

MNAR, missing not at random: missingness depends on the unobserved value. High earners decline to state income. No imputation method recovers the truth from the observed data alone, because the information required is precisely what is absent.

The uncomfortable part is that MAR and MNAR are not distinguishable from the data. It is an assumption, defended by knowledge of the collection process, and it should be stated rather than assumed by default.

What each imputation method actually assumes

Mean or median imputation replaces missing values with a constant. It preserves the mean, shrinks the variance (every imputed row sits exactly at the centre), and attenuates correlations with every other variable. Downstream standard errors are too small because the model treats imputed values as observed. It is fast and it is a biased estimator of everything except the mean.

Regression imputation predicts the missing value from the other columns. It respects MAR structure and produces values that are too good: they lie exactly on the fitted surface, so the imputed data are less variable than real data and correlations are inflated rather than attenuated. Stochastic regression imputation adds a residual draw, which fixes the variance.

Multiple imputation creates \(m\) complete datasets with different random draws, analyses each, and combines the results with Rubin's rules, where the total variance is the average within-imputation variance plus \((1 + 1/m)\) times the between-imputation variance. That second term is the whole point: it propagates the uncertainty about the imputed values into the final interval, which single imputation structurally cannot do. MICE, which cycles through columns imputing each from the others, is the standard implementation.

Model-native handling avoids the question. XGBoost learns a default direction per split for missing values, choosing whichever side reduces loss; LightGBM does similarly. This is often the best option for prediction, because it lets the model use missingness as information rather than erasing it.

The missingness indicator

Adding a binary was_missing column alongside the imputed value is cheap and frequently the highest-value step in the whole exercise. Under MNAR the fact of missingness carries signal that the imputed value destroys: a blank income field on a loan application is itself predictive, and no imputation can express that.

It also makes the model's dependence on missingness visible and auditable, which matters when a serving-time change in how often a field is populated shifts predictions and someone has to explain why.

When it breaks

Imputation fitted on the full dataset leaks. The imputer is a fitted transform. Fitting it before the train-test split means test-set values informed the training-set imputations. It belongs inside the cross-validation pipeline like a scaler.

Predictive performance and unbiased inference are different goals. For prediction, mean imputation plus an indicator often works well and multiple imputation is unnecessary complexity. For estimating a coefficient with an honest interval, single imputation understates uncertainty regardless of how good the imputer is. Choosing the method requires knowing which of the two you are doing.

The missingness pattern can change. A model trained when a field was 5% missing and served when it is 60% missing is evaluating a different feature. The imputed value is now most of the column, and the distribution of predictions moves without any drift in the underlying population.

MNAR has no data-only remedy. Sensitivity analysis, imputing under a range of assumed departures from MAR and reporting how the conclusion moves, is the honest treatment. A single imputed answer under an untested MAR assumption presents a choice as a fact.