Concept library
638 concepts across 13 domains and 52 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
15Search & Information Retrieval
10
Causal Inference & Experimentation
Telling correlation from cause, and measuring whether a change actually helped.
4tracks
17concepts
240cards
2.1hreading
Causal Foundations Potential outcomes, structural causal models, DAGs, confounding, colliders and the do-operator. 4 concepts · 58 cards
- 01 DAGs, Confounders and Colliders A causal graph turns "which variables should I control for" into a question with a mechanical answer, and shows why conditioning on the wrong variable creates bias rather than removing it.
- 02 Potential Outcomes and the Fundamental Problem A causal effect is a comparison of two outcomes for the same unit, only one of which is ever observed, which makes causal inference a missing-data problem rather than a modelling problem.
- 03 Simpson's Paradox and Choosing an Adjustment Set The same data can show an effect in every subgroup and the opposite effect in aggregate, and the arithmetic cannot tell you which is right; only the causal structure can.
- 04 The Do-Operator and Identification The distinction between conditioning on what you observed and intervening to set a value, and why identification is a question about the graph that must be settled before any estimation.
Online Experimentation Power, peeking, sample ratio mismatch, variance reduction, interference and long-term effects. 5 concepts · 64 cards
- 01 Sample Ratio Mismatch When the observed traffic split differs from the intended one, the randomisation is compromised and the effect estimate should be discarded rather than adjusted, because the cause is almost always a mechanism that also biases the metric.
- 02 Statistical Power and the Minimum Detectable Effect Sample size is decided by the smallest effect worth detecting, not by convention, and the fourth-power relationship between effect size and required traffic is why most product experiments are underpowered.
- 03 Interference and Network Effects in Experiments When one unit's treatment affects another unit's outcome, individual randomisation measures a quantity that is neither the treatment effect nor zero, and the standard designs trade bias against a large loss of power.
- 04 Peeking and Sequential Testing Fixed-sample p-values assume the sample size was chosen in advance, so continuously monitoring a dashboard and stopping at significance can inflate the false-positive rate several-fold.
- 05 Variance Reduction with CUPED and Stratification Regressing out pre-experiment behaviour removes variance that has nothing to do with the treatment, buying sensitivity without extra traffic, and the size of the gain is set by one correlation.
Observational Causal Methods Propensity scores, instrumental variables, difference-in-differences, regression discontinuity and synthetic control. 4 concepts · 58 cards
- 01 Difference-in-Differences and Parallel Trends Using a control group's change over time to estimate what the treated group's change would have been, and the untestable assumption that carries the entire argument.
- 02 Instrumental Variables Using a source of variation that affects treatment but has no other path to the outcome, which recovers a causal effect despite unmeasured confounding, for a subpopulation you cannot identify.
- 03 Propensity Scores and Matching Reducing a high-dimensional covariate vector to a single probability of treatment, which makes balancing tractable but does nothing about the confounders you did not measure.
- 04 Regression Discontinuity and Synthetic Control Two designs that manufacture a credible counterfactual, one from an arbitrary threshold in an assignment rule and one from a weighted combination of untreated units.
Policy Learning & Off-Policy Evaluation Heterogeneous treatment effects, uplift modelling, importance sampling estimators and doubly robust methods. 4 concepts · 60 cards
- 01 Doubly Robust Estimation Combining an outcome model with a propensity model so that the estimate stays consistent if either one is correct, and why the modern version adds cross-fitting to make that guarantee usable.
- 02 Heterogeneous Treatment Effects and Uplift Estimating who benefits rather than whether the average benefits, and why the target quantity is never observed for any individual, which breaks every standard model-selection habit.
- 03 Inverse Propensity Scoring for Off-Policy Evaluation Estimating how a new policy would have performed using only logs from an old one, by reweighting each logged decision by how much more likely the new policy was to make it.
- 04 Off-Policy Evaluation for Rankers and LLM Systems Applying counterfactual estimation where the action is a ranked list or a generated response, where the action space is effectively unbounded and the logging policy was never stochastic.