Self-Normalised and Clipped IPS Estimators

Four logged events, click-through rewards \(r = (1, 0, 1, 0)\), importance weights \(w = (0.5, 0.5, 8, 0.5)\). Plain IPS gives \(\hat{V} = (0.5 + 8)/4 = 2.125\). The reward is a click, bounded by 1, and the estimate says the new policy earns more than two clicks per impression. One event with a large weight has produced a number outside the feasible range. The inverse propensity scoring concept introduces the estimator and names its repairs; this one works through what each repair actually does to bias, variance and invariance.

Notation: logged tuples \((x_i, a_i, r_i)\) from logging policy \(\pi_0\), target policy \(\pi\), and weight \(w_i = \pi(a_i \mid x_i)/\pi_0(a_i \mid x_i)\).

Self-normalisation

The self-normalised estimator divides by the realised sum of weights instead of \(n\):

On the example, \(\hat{V}_{\text{SNIPS}} = 8.5/9.5 \approx 0.895\), inside \([0, 1]\) by construction, since it is a weighted average of observed rewards.

The justification is a control variate. Under common support, \(\mathbb{E}_{\pi_0}[w] = 1\) exactly, so the sample mean of weights \(S = \frac{1}{n}\sum_i w_i\) is a random variable with known expectation. Here \(S = 2.375\), which says the sample over-represents actions the target favours. SNIPS uses \(S\) as a multiplicative correction (Swaminathan and Joachims, 2015, The Self-Normalized Estimator for Counterfactual Learning, NeurIPS). The cost is a bias of order \(O(1/n)\), so the estimator is consistent rather than unbiased.

A second property matters more for learning than for evaluation. SNIPS is equivariant: add a constant \(c\) to every reward and the estimate moves by exactly \(c\). IPS moves by \(c \cdot S\). On the example, shifting rewards by 1 moves IPS from 2.125 to 4.5, and moves SNIPS from 0.895 to 1.895. An estimator whose ranking of two policies can flip when you relabel rewards from \(\{0,1\}\) to \(\{-1, 0\}\) is a poor objective to optimise.

Clipping

Clipping caps each weight at \(M\):

With \(M = 2\), the example gives \((0.5 + 2)/4 = 0.625\). Variance is now bounded by \(M^2\) times the reward second moment, and the bias is explicit:

For non-negative rewards clipping is always pessimistic, and it is most pessimistic for policies that differ most from logging, which is often a feature in deployment decisions. Clipping has a long production history, including Bing's ad placement analysis (Bottou et al., 2013, Counterfactual Reasoning and Learning Systems, JMLR 14); a truncation level that grows with \(n\) keeps the estimator consistent (Ionides, 2008, Truncated Importance Sampling, JCGS 17(2)). Choosing \(M\) is the whole difficulty: a fixed \(M\) chosen by eye has no guarantee, and the bias cannot be estimated from the clipped data alone.

The switch estimator

Clipping discards information in large-weight events. Wang, Agarwal and Dudík proposed replacing them with a reward model instead (Wang, Agarwal and Dudík, 2017, Optimal and Adaptive Off-policy Evaluation in Contextual Bandits, ICML, arXiv:1612.01205). With threshold \(\tau\) and reward model \(\hat{r}(x, a)\):

Actions whose weight would be small are evaluated by importance weighting; actions whose weight would explode are evaluated by the model. \(\tau = \infty\) recovers IPS, \(\tau = 0\) recovers the direct method. The paper also proves that without a consistent reward model, IPS and doubly robust estimators are minimax optimal, so the gains from switching come entirely from the model being useful where weights are large. \(\tau\) is chosen by minimising an estimate of variance plus a bound on squared bias.

When it breaks

No reweighting creates support. All three estimators assume \(\pi_0(a \mid x) > 0\) wherever \(\pi(a \mid x) > 0\). Under deficient support, \(\mathbb{E}[w] < 1\), and SNIPS silently renormalises over the actions it has seen, which overstates the value if the unseen actions are worse.

Self-normalisation is not the last word. Jeunen and Gupta prove that IPS with an optimally chosen additive baseline asymptotically dominates SNIPS in mean squared error, and that SNIPS is equivalent to one particular, generally sub-optimal, baseline (Jeunen and Gupta, 2026, Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation, arXiv:2602.14914). The result is recent, posted in February 2026, and whether it displaces self-normalisation in practice is still open as of September 2026.

Tuning on the evaluation data biases the evaluation. Picking \(M\) or \(\tau\) to make the estimate look stable, then reporting that estimate, is a garden of forking paths. Su and colleagues frame weight clipping and shrinkage as optimising an explicit MSE bound, which at least makes the choice principled (Su et al., 2020, Doubly Robust Off-Policy Evaluation with Shrinkage, arXiv:1907.09623).

Effective sample size still rules. On the example, \((\sum w)^2 / \sum w^2 = 90.25 / 64.75 \approx 1.4\). Four events carry the information of about one and a half. No estimator in this family fixes that; they only choose how to be wrong about it.