Tagged “causal-inference”

3 posts.

All Model Architecture15 Training & Alignment18 Inference & Serving16 Agents & Orchestration11 Reasoning & Evaluation12 Safety, Security & Governance3 Platforms & Practice15

Platforms & Practice 27 min

Four Ways an A/B Test Lies: Power, Peeking, Sample Ratio Mismatch and Interference

At Microsoft, roughly one third of tested ideas move the metric in the intended direction. That number is only meaningful if the tests are trustworthy, and the four failure modes that destroy trustworthiness all produce a clean, …

experimentation ab-testing causal-inference statistics ∑ ◫

Platforms & Practice 24 min

Reading Its Own Handwriting: Position Bias, Feedback Loops and Counterfactual Correction

Move a document from rank 5 to rank 1 and its click-through rate jumps without its relevance changing at all. Train on those clicks and the model learns position, which the previous ranker chose, which means it learns to imitate …

ranking recommender-systems learning-to-rank counterfactual ∑ ◫

Platforms & Practice 25 min

The Controls That Make It Worse: Colliders, Mediators and Why Adjusting for Everything Is Wrong

Open surgery beat percutaneous nephrolithotomy on small kidney stones, 93% to 87%. It beat it on large stones too, 73% to 69%. Pooled across both, it lost, 78% to 83%. The arithmetic is correct in all three statements, and no amo…

causal-inference statistics confounding dags ∑ ◫