Concept library
1134 concepts across 20 domains and 101 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
13Generative Modelling Beyond Transformers
14Efficiency, Compression & Edge AI
15Search & Information Retrieval
16Data & Feature Engineering
17MLOps & Platform Engineering
18Security, Privacy & Adversarial ML
19Governance, Risk & Responsible AI
20Human-AI Interaction, Product & Economics
10
Causal Inference & Experimentation
Telling correlation from cause, and measuring whether a change actually helped.
4tracks
28concepts
557cards
3.6hreading
Causal Foundations Potential outcomes, structural causal models, DAGs, confounding, colliders and the do-operator. 7 concepts · 137 cards
- 01 DAGs, Confounders and Colliders A causal graph turns "which variables should I control for" into a question with a mechanical answer, and shows why conditioning on the wrong variable creates bias rather than removing it.
- 02 Identification Assumptions: Exchangeability, Positivity and Consistency The three conditions that let an observed conditional mean stand in for an unobserved potential outcome, how each one enters the identification proof, and which of them data can and cannot check.
- 03 Potential Outcomes and the Fundamental Problem A causal effect is a comparison of two outcomes for the same unit, only one of which is ever observed, which makes causal inference a missing-data problem rather than a modelling problem.
- 04 Simpson's Paradox and Choosing an Adjustment Set The same data can show an effect in every subgroup and the opposite effect in aggregate, and the arithmetic cannot tell you which is right; only the causal structure can.
- 05 Mediation Analysis: Direct and Indirect Effects Splitting a total effect into the part that runs through a mediator and the part that does not requires counterfactuals that mix two worlds, which is why the decomposition needs assumptions that even a randomised experiment cannot deliver.
- 06 Structural Causal Models and Counterfactuals A structural causal model assigns every variable a mechanism and an exogenous noise term, which is exactly the extra structure needed to answer "what would have happened to this unit", the question neither correlations nor interventions can answer.
- 07 The Do-Operator and Identification The distinction between conditioning on what you observed and intervening to set a value, and why identification is a question about the graph that must be settled before any estimation.
Online Experimentation Power, peeking, sample ratio mismatch, variance reduction, interference and long-term effects. 7 concepts · 142 cards
- 01 Overall Evaluation Criterion and Guardrail Metrics Choosing the metric an experiment is judged on, why the obvious business metrics make bad ones, and how guardrail metrics with non-inferiority tests protect everything the goal metric does not measure.
- 02 Sample Ratio Mismatch When the observed traffic split differs from the intended one, the randomisation is compromised and the effect estimate should be discarded rather than adjusted, because the cause is almost always a mechanism that also biases the metric.
- 03 Statistical Power and the Minimum Detectable Effect Sample size is decided by the smallest effect worth detecting, not by convention, and the fourth-power relationship between effect size and required traffic is why most product experiments are underpowered.
- 04 Interference and Network Effects in Experiments When one unit's treatment affects another unit's outcome, individual randomisation measures a quantity that is neither the treatment effect nor zero, and the standard designs trade bias against a large loss of power.
- 05 Novelty, Primacy and Long-Term Effects Treatment effects that grow or decay as users learn, how to detect and measure that learning, and the two ways to estimate a long-term effect without waiting years, long-running holdouts and surrogate indices.
- 06 Peeking and Sequential Testing Fixed-sample p-values assume the sample size was chosen in advance, so continuously monitoring a dashboard and stopping at significance can inflate the false-positive rate several-fold.
- 07 Variance Reduction with CUPED and Stratification Regressing out pre-experiment behaviour removes variance that has nothing to do with the treatment, buying sensitivity without extra traffic, and the size of the gain is set by one correlation.
Observational Causal Methods Propensity scores, instrumental variables, difference-in-differences, regression discontinuity and synthetic control. 7 concepts · 137 cards
- 01 Difference-in-Differences and Parallel Trends Using a control group's change over time to estimate what the treated group's change would have been, and the untestable assumption that carries the entire argument.
- 02 G-Computation and Regression Adjustment Turning an outcome model into a causal effect by predicting every unit's outcome under each treatment and averaging, and why that is not the same as reading a coefficient off the regression.
- 03 Sensitivity Analysis and E-Values Asking how strong an unmeasured confounder would have to be to explain away an observational result, and the three standard ways to quantify that, from Rosenbaum's gamma to the E-value to partial R-squared bounds.
- 04 Double Machine Learning Using arbitrary machine-learning models to adjust for high-dimensional confounders while keeping a root-n, asymptotically normal estimate of the causal parameter, through orthogonal scores and cross-fitting.
- 05 Instrumental Variables Using a source of variation that affects treatment but has no other path to the outcome, which recovers a causal effect despite unmeasured confounding, for a subpopulation you cannot identify.
- 06 Propensity Scores and Matching Reducing a high-dimensional covariate vector to a single probability of treatment, which makes balancing tractable but does nothing about the confounders you did not measure.
- 07 Regression Discontinuity and Synthetic Control Two designs that manufacture a credible counterfactual, one from an arbitrary threshold in an assignment rule and one from a weighted combination of untreated units.
Policy Learning & Off-Policy Evaluation Heterogeneous treatment effects, uplift modelling, importance sampling estimators and doubly robust methods. 7 concepts · 141 cards
- 01 Counterfactual Risk Minimisation Learning a new policy directly from logged bandit feedback by minimising an importance-weighted risk plus a variance penalty, and why naive IPS objectives overfit to the logging policy's choices rather than to the rewards.
- 02 Doubly Robust Estimation Combining an outcome model with a propensity model so that the estimate stays consistent if either one is correct, and why the modern version adds cross-fitting to make that guarantee usable.
- 03 Heterogeneous Treatment Effects and Uplift Estimating who benefits rather than whether the average benefits, and why the target quantity is never observed for any individual, which breaks every standard model-selection habit.
- 04 Inverse Propensity Scoring for Off-Policy Evaluation Estimating how a new policy would have performed using only logs from an old one, by reweighting each logged decision by how much more likely the new policy was to make it.
- 05 Off-Policy Evaluation for Rankers and LLM Systems Applying counterfactual estimation where the action is a ranked list or a generated response, where the action space is effectively unbounded and the logging policy was never stochastic.
- 06 Self-Normalised and Clipped IPS Estimators The three standard repairs to inverse propensity scoring, self-normalisation, weight clipping and the switch estimator, what bias each buys its variance reduction with, and how to choose between them.
- 07 Slate Off-Policy Evaluation Evaluating ranked lists and page layouts from logs when the number of possible slates is astronomically large, by assuming structure in how slate rewards decompose, with the pseudoinverse estimator as the central example.