Concept library
1129 concepts across 20 domains and 101 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
13Generative Modelling Beyond Transformers
14Efficiency, Compression & Edge AI
15Search & Information Retrieval
16Data & Feature Engineering
17MLOps & Platform Engineering
18Security, Privacy & Adversarial ML
19Governance, Risk & Responsible AI
20Human-AI Interaction, Product & Economics
09
Classical ML & Statistical Learning
The statistics and non-neural models that still decide most production predictions.
6tracks
43concepts
885cards
5.4hreading
Statistical Inference Estimators, likelihood, the bootstrap, hypothesis testing, multiplicity and what a confidence interval really claims. 7 concepts · 149 cards
- 01 Confidence Intervals and Coverage What the 95% in a 95% interval refers to, why the Wald interval for a proportion is badly behaved near zero, and how a confidence interval differs from the credible interval people usually think they are reading.
- 02 Estimators, Bias, Variance and Consistency What makes one estimator better than another, why the mean squared error splits cleanly into bias squared plus variance, and why an unbiased estimator is frequently the wrong thing to want.
- 03 Hypothesis Tests, Error Types and the Neyman–Pearson Lemma A hypothesis test is a decision rule with two error rates that trade against each other, and the Neyman–Pearson lemma says the likelihood ratio is the rule that buys the most power for a fixed false-positive rate.
- 04 P-Values, Multiplicity and the Garden of Forking Paths What a p-value actually claims, why running twenty tests guarantees a false discovery, and the difference between controlling the family-wise error rate and controlling the false discovery rate.
- 05 Sampling Distributions and the Central Limit Theorem Every standard error and every normal-theory interval is a statement about a distribution you never observe, and the central limit theorem is a promise about that distribution whose speed of delivery depends on skewness, tails and dependence.
- 06 The Bootstrap and Resampling Resampling the data you have to simulate the sampling distribution you never observed, why the percentile interval is not always the right one, and the specific statistics for which the bootstrap silently fails.
- 07 Likelihood Ratio, Wald and Score Tests Three asymptotically equivalent ways to test a parameter from the log-likelihood, which measure the vertical drop, the horizontal distance and the slope at the null, and which can disagree wildly in exactly the samples where the answer matters.
Classical Supervised Learning Linear and logistic regression, regularisation, margins, kernels, and the geometry underneath them. 8 concepts · 156 cards
- 01 Linear Regression as Projection Least squares is orthogonal projection onto the column space of the design matrix, which explains the normal equations, the meaning of residuals, and why nobody who ships numerical code inverts the matrix.
- 02 Logistic Regression and the Log-Odds Why classification is modelled on the log-odds scale rather than the probability scale, why there is no closed-form solution, and what perfect separation does to the coefficients.
- 03 k-Nearest Neighbours and the Curse of Dimensionality The simplest classifier there is comes with a strong guarantee in low dimensions and a quiet collapse in high ones, because neighbourhoods stop being local and distances stop discriminating.
- 04 Cross-Validation and Model Selection Cross-validation estimates how well a learning procedure generalises, not how well your fitted model will, and the moment you use it to pick among models its estimate of the winner is optimistically biased.
- 05 Naive Bayes and Generative Classifiers Naive Bayes models how each class generates its features under an assumption that is almost always false, and still classifies well, because getting the ranking right is far easier than getting the probabilities right.
- 06 Ridge, Lasso and Elastic Net Why an $\ell_1$ penalty produces exact zeros while $\ell_2$ only shrinks, what each does to correlated features, and why regularisation is a bias-variance trade rather than a way to fix a bad model.
- 07 Margins, Kernels and the Support Vector Machine How maximising the distance to the nearest point gives a classifier that depends on a handful of examples, and how the dual formulation lets you work in an infinite-dimensional feature space without ever visiting it.
- 08 Probability Calibration: Platt Scaling and Isotonic Regression A classifier can rank perfectly and still report probabilities that are wrong by a factor of three, and post-hoc calibration fixes the numbers without touching the ranking, at a cost in held-out data that depends on which method you choose.
Trees & Ensembles Impurity splitting, bagging, random forests, gradient boosting, and the engineering inside XGBoost and LightGBM. 7 concepts · 145 cards
- 01 Decision Trees and Impurity Splitting How a greedy search over axis-aligned splits builds a piecewise-constant function, why Gini and entropy almost never disagree, and the specific structural biases that make a single tree unstable.
- 02 Bagging and Random Forests Why averaging unstable models reduces variance, why bootstrap sampling alone is not enough, and what the extra feature subsampling in a random forest is actually buying.
- 03 Stacking and Blending Stacking learns how to combine several models from their out-of-fold predictions, and nearly all of its value and all of its danger come from how those predictions are produced.
- 04 Why Trees Still Beat Deep Nets on Tabular Data The three inductive biases that separate tree ensembles from neural networks on tabular problems, and the specific dataset conditions under which the ordering has been observed to flip.
- 05 Gradient Boosting as Functional Gradient Descent Boosting is gradient descent in function space, where each new tree approximates the negative gradient of the loss, which is what lets a single algorithm fit any differentiable objective.
- 06 Histogram Gradient Boosting: The Engineering Inside XGBoost, LightGBM and CatBoost The three dominant boosting libraries share one objective and differ in how they search for splits, grow trees and handle leakage, and those engineering choices decide speed, memory and which datasets each one overfits.
- 07 TreeSHAP and Feature Importance Impurity, permutation and SHAP importance answer three different questions about a tree ensemble, each with a known bias, and TreeSHAP made the game-theoretic one exact and fast without settling which question you should be asking.
Unsupervised Learning Clustering, mixture models and EM, PCA and SVD, manifold embeddings, and density estimation. 7 concepts · 145 cards
- 01 K-Means and Its Assumptions Lloyd's algorithm is coordinate descent on a squared-error objective, which explains why it always converges, why it converges to the wrong answer without careful seeding, and the exact cluster shapes it cannot represent.
- 02 Density-Based Clustering with DBSCAN and HDBSCAN DBSCAN defines a cluster as a connected region of high point density and labels everything else as noise, and HDBSCAN removes its single density threshold by building the whole hierarchy of thresholds and keeping the most persistent clusters.
- 03 Hierarchical Clustering and Linkage Agglomerative clustering builds a full tree of merges instead of one partition, and the linkage rule that scores each merge decides whether the tree finds crescents, chains through noise, or reproduces k-means.
- 04 PCA, SVD and Whitening Why maximising retained variance and minimising reconstruction error give the same answer, how the SVD computes it without ever forming a covariance matrix, and what whitening destroys.
- 05 UMAP, t-SNE and What They Distort Neighbour embeddings optimise local neighbourhood preservation and nothing else, which makes cluster separation, cluster size and inter-cluster distance in the resulting picture largely uninterpretable.
- 06 Clustering Validation: Silhouette, Gap, ARI and Choosing k Internal indices like silhouette and the gap statistic score a clustering against its own geometric ideal, external indices like ARI and NMI need chance correction to mean anything, and Kleinberg's impossibility theorem explains why no index can settle what the right clustering is.
- 07 Gaussian Mixtures and the EM Algorithm How treating the cluster label as a latent variable turns an intractable likelihood into two closed-form steps, why the likelihood is guaranteed to increase, and why it is unbounded above.
Bayesian Methods Priors and posteriors, MCMC and HMC, variational inference, Gaussian processes and model comparison. 7 concepts · 145 cards
- 01 Hierarchical Models and Partial Pooling A hierarchical model lets groups borrow strength from each other by learning how different they are, which turns the choice between separate estimates and one pooled estimate into a parameter the data decides.
- 02 Posterior Predictive Checks and the Bayesian Workflow A posterior predictive check simulates replicated data from the fitted model and asks whether they look like the real data in the ways that matter, and it is one step in a workflow that also checks priors, computation and the model's ability to recover known parameters.
- 03 Priors, Conjugacy and the Posterior How a prior functions as pseudo-data, why conjugate families make the posterior a closed-form update, and why "uninformative" priors are informative on some scale.
- 04 Bayesian Model Comparison: Bayes Factors, PSIS-LOO and WAIC Bayes factors compare models by how well their priors predicted the data and are acutely sensitive to prior width, while PSIS-LOO and WAIC estimate out-of-sample predictive accuracy from the posterior and come with diagnostics that say when to distrust them.
- 05 Gaussian Processes Placing a prior over functions rather than parameters, which gives exact posterior uncertainty that grows away from the data, at a cubic cost that determines where they are usable.
- 06 MCMC and Hamiltonian Monte Carlo Why sampling from an unnormalised posterior is possible at all, why random-walk proposals fail in high dimensions, and how using gradient information turns a random walk into directed motion.
- 07 Variational Inference and the ELBO Turning integration into optimisation by fitting a tractable distribution to the posterior, and the specific bias that comes from minimising the reverse KL divergence.
Feature Engineering Encoding, missingness, selection, target leakage, and the train-serve skew that eats offline gains. 7 concepts · 145 cards
- 01 Feature Scaling and Transformations Standardisation, power transforms and quantile maps change what distance, penalties and gradients see, so they matter enormously for some model families and not at all for others, and transforming the target quietly changes what the model predicts.
- 02 Categorical Encoding Why one-hot encoding breaks down at high cardinality, how target encoding trades that for a leakage risk it must then defend against, and what each choice assumes about unseen categories.
- 03 Feature Selection: Filter, Wrapper and Embedded Methods Filter, wrapper and embedded methods trade cost against how much of the model they consult, and all three produce wildly optimistic accuracy if the selection step happens outside the cross-validation loop.
- 04 Missing Data Mechanisms Whether imputation is safe depends on why the value is absent, and the three-way distinction between MCAR, MAR and MNAR decides which methods are valid and which quietly bias the result.
- 05 Target Leakage The failure mode where a feature encodes information unavailable at prediction time, why cross-validation cannot detect it, and the three structural forms it takes.
- 06 Train-Serve Skew The gap between how a feature is computed in a training pipeline and how it is computed in a serving path, which silently degrades a model that was never wrong in offline evaluation.
- 07 Class Imbalance, Resampling and Reweighting Rebalancing classes by oversampling, SMOTE or class weights mostly moves the decision threshold while destroying probability calibration, and the evidence suggests strong learners gain little from it compared with choosing the threshold from actual costs.