Tagged “causal-inference”

Reasoning & Evaluation 24 min

Double Machine Learning: How to Use Flexible Models for Causal Estimates Without Inheriting Their Bias

Plug a random forest into a causal regression and the confidence interval tightens around the wrong number, because regularisation bias shrinks more slowly than the standard error. Double machine learning fixes this with two devi…

observational-causal-methods causal-inference confounding estimation ∑ ◫

Platforms & Practice 27 min

Four Ways an A/B Test Lies: Power, Peeking, Sample Ratio Mismatch and Interference

At Microsoft, roughly one third of tested ideas move the metric in the intended direction. That number is only meaningful if the tests are trustworthy, and the four failure modes that destroy trustworthiness all produce a clean, …

experimentation ab-testing causal-inference statistics ∑ ◫

Training & Alignment 24 min

Learning From the Log: Off-Policy Policy Learning, From IPS to Counterfactual Risk Minimisation

An unbiased estimate of every policy's value does not give you an unbiased choice of policy. The moment an optimiser searches over importance-weighted estimates, it goes looking for the estimator's noise, and the history of learn…

policy-learning-and-ope off-policy-evaluation counterfactual importance-sampling ∑ ◫

Platforms & Practice 24 min

Reading Its Own Handwriting: Position Bias, Feedback Loops and Counterfactual Correction

Move a document from rank 5 to rank 1 and its click-through rate jumps without its relevance changing at all. Train on those clicks and the model learns position, which the previous ranker chose, which means it learns to imitate …

ranking recommender-systems learning-to-rank counterfactual ∑ ◫

Reasoning & Evaluation 29 min

The Control Group Is Disappearing: Measuring AI-Assisted Development

Four credible randomised trials of AI coding tools report effects from a 19 percent slowdown to a 56 percent speedup. The newest problem is worse than the spread: developers now refuse to be randomised into the no-AI arm, and 30 …

evaluation measurement experimentation causal-inference ∑ ◫

Platforms & Practice 25 min

The Controls That Make It Worse: Colliders, Mediators and Why Adjusting for Everything Is Wrong

Open surgery beat percutaneous nephrolithotomy on small kidney stones, 93% to 87%. It beat it on large stones too, 73% to 69%. Pooled across both, it lost, 78% to 83%. The arithmetic is correct in all three statements, and no amo…

causal-inference statistics confounding dags ∑ ◫

Reasoning & Evaluation 24 min

The Ladder of Causation: Why No Amount of Observational Data Climbs It Alone

In 2023 GPT-4 scored 97% on a classic cause-and-effect benchmark and 62% on one that hands it the causal graph and asks it to compute. Both results fit a theorem proved in 2020: data from one rung of Pearl's ladder almost never d…

causal-foundations causal-inference counterfactual dags ∑ ◫

Reasoning & Evaluation 24 min

What a Feature Attribution Can and Cannot Tell You: SHAP, LIME and the Explanation Gap

Add a column the model never reads and SHAP can hand it more than a quarter of the credit for a decision. That is not a library bug: a Shapley attribution answers a question you chose, often without noticing, and a 2024 PNAS resu…

transparency-and-documentation interpretability responsible-ai causal-inference ∑ ◫