Doubly Robust Estimation

Two estimators, two failure modes. Regression adjustment models the outcome and is consistent if that model is right. Inverse propensity weighting models treatment assignment and is consistent if that model is right. Getting either exactly right is unlikely, and choosing between them is a bet on which you can specify better.

The doubly robust estimator takes both and combines them so that being right about either one suffices:

The first two terms are the outcome-model estimate. The remaining terms are an IPW correction applied to the outcome model's residuals, not to the outcomes themselves.

Why "either one" works

Trace the two cases. If the outcome models \(\hat{\mu}_t\) are correct, their residuals have mean zero given \(x\), so the correction terms vanish in expectation regardless of how wrong \(\hat{e}\) is. If instead the propensity model is correct, the weighting is valid and the whole expression reduces to an unbiased IPW estimator with the outcome model acting as a variance-reducing control variate. Two independent routes to consistency, and only one needs to hold.

The name is slightly generous. Double robustness protects against misspecification of one model, not both, and the estimator inherits the combined error when both are somewhat wrong, which is the realistic case. It is a hedge, not a guarantee.

Cross-fitting, and why it matters more than it sounds

Plugging flexible machine learning models into \(\hat{\mu}\) and \(\hat{e}\) appears to be the obvious modern upgrade, and doing it naively fails. Regularisation biases the nuisance estimates, and using the same data to fit the nuisances and evaluate the estimator makes the errors correlated, so the bias does not wash out and the confidence intervals do not cover.

Double machine learning fixes both (Chernozhukov et al., 2018, Econometrics Journal 21(1), C1-C68). The score is Neyman-orthogonal, meaning its derivative with respect to the nuisance parameters is zero at the truth, so first-order errors in \(\hat{\mu}\) and \(\hat{e}\) do not propagate into the causal estimate. And cross-fitting estimates the nuisances on one fold and evaluates the score on another, removing the own-observation dependence.

The resulting condition is a product: the estimator is \(\sqrt{n}\)-consistent and asymptotically normal provided the product of the two nuisance error rates converges faster than \(n^{-1/2}\). Each model may converge at \(n^{-1/4}\), well within reach for random forests or gradient boosting, and the causal estimate still attains the parametric rate. That result is what made flexible ML usable for causal estimation with valid intervals.

When it breaks

Extreme propensities dominate everything. The correction terms divide by \(\hat{e}\) and \(1 - \hat{e}\), so a score near 0 or 1 produces an enormous residual correction. Doubly robust estimators can be less stable than plain regression adjustment in poor-overlap regions, which is the opposite of the intuition that combining methods is safer. Trimming and stabilised weights remain necessary.

Both models wrong is the normal case. The theorem covers one being right. In practice neither is exactly correct, and the bias depends on both errors and their interaction. The honest framing is that DR reduces sensitivity to specification, not that it removes it.

Cross-fitting is not optional. A doubly robust estimator with machine learning nuisances fitted on the full sample gives intervals that are too narrow and estimates that carry regularisation bias. This is a common implementation shortcut and it removes the property the method was chosen for.

The estimand still depends on ignorability. Everything here assumes no unmeasured confounding. Double robustness is insurance against getting the functional form wrong, and it offers nothing against having omitted a confounder. It is easy to read the word "robust" as covering more than it does.