Concept library
1015 concepts across 20 domains and 101 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
13Generative Modelling Beyond Transformers
14Efficiency, Compression & Edge AI
15Search & Information Retrieval
16Data & Feature Engineering
17MLOps & Platform Engineering
18Security, Privacy & Adversarial ML
19Governance, Risk & Responsible AI
20Human-AI Interaction, Product & Economics
01
Foundations
The mathematics and neural-network mechanics everything else assumes.
4tracks
63concepts
707cards
8.4hreading
Mathematical Foundations Linear algebra, probability, calculus and optimisation — the machinery every model is built on. 16 concepts · 223 cards
- 01 Calculus and Gradients Partial derivatives, the chain rule as the engine of backprop, why second-order methods are rare in deep learning, and what gradient clipping actually does.
- 02 Fourier Analysis for Neural Models How the convolution theorem turns quadratic sequence mixing into n log n, why MLPs refuse to learn high frequencies, and why rotary position encoding is a frequency argument wearing a rotation costume.
- 03 Linear Algebra for ML Vector spaces, matrix decompositions, and why low-rank structure underlies LoRA, PCA, and the quantisation tricks that make modern LLMs cheap to serve.
- 04 Numerical Computation Gotchas Catastrophic cancellation, the log-sum-exp trick, mixed-precision training, the determinism tax, and how to actually debug a NaN in a 70B model.
- 05 Probability and Information Theory Distributions, expectations, entropy, KL, and why softmax + cross-entropy is the canonical pair that secretly underlies almost every LLM loss.
- 06 Concentration Inequalities and Generalisation Bounds The tools that turn "the average of many random things is close to its mean" into explicit numbers, how they build classical generalisation bounds, and why those bounds are vacuous for deep networks.
- 07 Fisher Information and Natural Gradient Why the steepest descent direction depends on the coordinate system you happened to choose, how the Fisher matrix fixes that, and which popular optimisers are approximating it badly.
- 08 Group Equivariance and Symmetry When a task has a symmetry, building it into the architecture instead of learning it from data cuts sample complexity, and the group-theoretic formulation says exactly how to do that.
- 09 Lagrangian Duality and Constrained Optimisation How a constraint becomes a penalty with a price attached, why the KL-regularised objective at the heart of RLHF has a closed-form solution, and where duality quietly fails.
- 10 Loss Landscape Geometry and Mode Connectivity Independently trained networks are not isolated in separate valleys; they are connected by low-loss paths, and after undoing permutation symmetry they are often in the same basin.
- 11 Optimal Transport and Wasserstein Distances Why KL divergence is useless between distributions that do not overlap, how the cost of moving mass gives a metric that is not, and what entropic regularisation trades away to make it computable.
- 12 Optimisation Theory Convexity, why SGD finds good solutions on non-convex losses, saddle points at scale, momentum as a damped oscillator, and learning-rate schedules as implicit regularisation.
- 13 SGD as a Stochastic Differential Equation Treating mini-batch gradient descent as a noisy continuous-time process explains the linear scaling rule, why the learning-rate-to-batch-size ratio is the real hyperparameter, and where the analogy fails.
- 14 Spectral Analysis of Weight Matrices What the singular values of a weight matrix control, why random matrix theory predicts the spectrum at initialisation, and how spectral thinking connects initialisation, LoRA and modern optimisers.
- 15 Statistical Learning Theory Primer Bias-variance, PAC-learning, VC dimension, why deep nets break classical generalisation bounds, double descent, and what scaling laws are actually saying.
- 16 The Manifold Hypothesis and Intrinsic Dimension Why high-dimensional data concentrates on a low-dimensional surface, how to estimate that surface's dimension from samples, and why the number predicts how much data a model will need.
Tensors & Neural Plumbing Shapes, matmuls, forward and backward passes, parameter counts, memory footprints. 18 concepts · 211 cards
- 01 Broadcasting and Vectorisation Broadcasting is the rule that lets a bias vector add to every row of a batch without an explicit loop, and understanding its shape-matching logic prevents the class of bugs that produce wrong answers without an error.
- 02 Matrix Multiplication: The Core Operation Nearly every FLOP a transformer spends is a matrix multiplication; understanding its shape rule and its cost is the single most load-bearing piece of maths in deep learning.
- 03 Tensors, Shapes, and Batching Every number a transformer touches lives inside a tensor with a fixed shape; learning to read and predict those shapes is the fastest way to stop being confused by model code.
- 04 Why Non-Linearity Matters Stack any number of linear layers and the result is still one linear layer; the humble activation function is the only thing standing between a transformer and a glorified matrix multiplication.
- 05 Checkpoint Formats and State Dicts A checkpoint is a dictionary from parameter names to tensors plus a pile of implicit assumptions about dtype, sharding, and key naming, and every one of those assumptions is somewhere a load goes wrong.
- 06 Counting Transformer Parameters A transformer's parameter count is not a mysterious headline number, it is the sum of a handful of matrix shapes multiplied out, and knowing the formula lets you sanity-check any model card in seconds.
- 07 Einsum and Tensor Contractions Einstein summation notation expresses matmuls, batched matmuls, and attention itself as one uniform pattern, and reading it fluently is the fastest way to understand what a line of unfamiliar model code actually computes.
- 08 Forward-Mode and Reverse-Mode Autodiff Backpropagation is one of two ways to apply the chain rule mechanically, and the choice between them is decided by a single number, the ratio of inputs to outputs, which is why training uses reverse mode and Hessian-vector products use both.
- 09 Padding, Masks, and Variable-Length Batching Real batches contain sequences of different lengths, and the three ways of handling that difference, padding, packing, and varlen kernels, have wildly different costs and each has a signature bug.
- 10 Tensor Memory Layout and Contiguity A tensor is a pointer, a shape, and a stride tuple; transposes and slices change only the metadata, which is why some reshapes are free, some silently copy gigabytes, and a permute in the wrong place can halve your throughput.
- 11 The Backward Pass and Gradient Flow Backpropagation through a transformer is the forward pass run in reverse with the chain rule attached; seeing which operations preserve gradient magnitude and which shrink it explains why architecture choices exist.
- 12 The Forward Pass End to End Trace one token's numbers from embedding table to output logits and every "mysterious" transformer component turns out to be a shape-preserving or shape-mixing step in a fixed pipeline.
- 13 Views, Aliasing, and In-Place Operations A transpose returns a new tensor object that shares its old storage, which is why in-place edits leak across variables you thought were separate and why autograd raises "a variable needed for gradient computation has been modified".
- 14 Accumulation Precision and Mixed-Dtype Plumbing Storing weights in 16 bits is safe because almost nothing is actually computed in 16 bits — matmuls accumulate in fp32, reductions run in fp32, and the master copy is fp32, and every documented failure of low-precision training is one of these rules being broken.
- 15 Batch Invariance and Numerical Reproducibility Temperature zero does not give the same answer twice, and the reason is neither random seeds nor GPU atomics; it is that reduction kernels change their split strategy with batch shape, so your logits depend on which other requests happened to be in flight.
- 16 Signal Propagation and Initialisation How a network's weights are initialised decides, before a single gradient step, whether activations and gradients stay in a trainable range or collapse to zero or explode across depth.
- 17 Training Memory Footprint A model's weights are the smallest part of its training memory bill; optimiser state, gradients, and activations usually cost several times more, and knowing the breakdown explains why training needs far more memory than inference.
- 18 muP and Hyperparameter Transfer Under standard parametrisation the best learning rate drifts as a model gets wider, so every scale-up is a fresh search; muP rescales initialisation, learning rates, and multipliers per layer so the optimum stays put and can be tuned on a small proxy model.
Deep Learning Building Blocks Convolutions, recurrence, normalisation, activations, optimisers and regularisation. 13 concepts · 112 cards
- 01 Dropout and Modern Regularisation Why dropout was the dominant regulariser for a decade and why modern LLM training mostly skips it in favour of letting data do the work.
- 02 Activation Functions Why depth is meaningless without a nonlinearity, and how the field moved from saturating sigmoids through ReLU and GELU to the gated SwiGLU that sits in almost every modern transformer feed-forward block.
- 03 Backpropagation and Automatic Differentiation How reverse-mode autodiff turns the chain rule into an efficient gradient algorithm, and the design choices PyTorch and JAX make to implement it at scale.
- 04 Convolutional Neural Networks Why weight sharing and local receptive fields make CNNs the right inductive bias for images, and where ViTs took over.
- 05 Generative Adversarial Networks The minimax game that dominated image generation for six years, why its training instability and mode collapse are structural rather than incidental, and what it left behind after diffusion overtook it.
- 06 Graph Neural Networks and Message Passing How one aggregate-and-update primitive covers most of the GNN literature, what the Weisfeiler-Lehman test says about its expressive ceiling, and why oversmoothing and oversquashing put a hard cap on depth.
- 07 Normalisation: BatchNorm, LayerNorm, RMSNorm Why normalisation accelerates training, why transformers use LayerNorm instead of BatchNorm, and why RMSNorm is now the default in Llama-class models.
- 08 Optimisers: SGD, Adam, AdamW, Lion How the standard optimiser stack evolved from plain SGD through Adam to memory-cheaper variants like Lion and Muon, and which learning-rate schedules actually work at scale.
- 09 Recurrent Networks: RNN, LSTM, GRU How gating fixed the vanishing-gradient problem in RNNs, and why transformers displaced them everywhere except streaming and on-device workloads.
- 10 Residual Connections and Skip Paths Why adding the input back to a layer's output is what makes networks of dozens or hundreds of layers trainable, and how the same trick underpins every modern transformer.
- 11 Self-Supervised Contrastive Learning How InfoNCE turns "these two crops came from the same photo" into a training signal strong enough to match supervised pretraining, and why every method in this family is fundamentally an anti-collapse mechanism.
- 12 State Space Models and Selective SSMs How a linear recurrence with a structured state matrix reaches transformer-level language modelling at linear cost and constant-memory decoding, what selectivity added, and the copying tasks that still expose the gap.
- 13 Variational Autoencoders and the ELBO How maximising an intractable data likelihood turns into maximising a tractable lower bound, why the reparameterisation trick is what makes that bound differentiable, and what posterior collapse costs you.
Information Theory for Language Entropy, cross-entropy, KL, perplexity, calibration, and language modelling as compression. 16 concepts · 161 cards
- 01 Log-Probs and Token Probabilities What the log-probability numbers an API actually returns mean, how to turn them back into probabilities and perplexity, and the practical uses (reranking, confidence checks, hallucination flags) that make them worth reading instead of skipping.
- 02 N-Gram Models and Smoothing The pre-neural language model that ruled for three decades, why counting words breaks the moment you hit a sequence you have never seen, and the smoothing tricks invented to patch that hole.
- 03 Perplexity - Measuring a Language Model The oldest language-model metric, what "perplexity 12" actually means, and why you can never compare it across two models with different tokenisers.
- 04 Bits-per-Byte The tokenizer-invariant way to report a language model's compression performance, how to compute it from cross-entropy, and why serious scaling-law and benchmark papers report it instead of perplexity.
- 05 Cross-Entropy and KL Divergence The precise relationship between cross-entropy, entropy, and KL divergence, why minimising one is minimising "excess bits," and why forward and reverse KL pull a model in opposite directions.
- 06 Entropy and Surprise Why "surprising" has an exact numerical meaning, how -log p(x) turns probability into a unit of information, and what a language model's entropy floor tells you about a piece of text.
- 07 Maximum-Likelihood Estimation The estimation principle underneath every LLM's training objective, why "maximise probability of the data" and "minimise cross-entropy" are the exact same optimisation, and what changes when you add a prior.
- 08 Calibration of Language Models What it means for a model's stated confidence to be trustworthy, why pretrained next-token probabilities start out well calibrated and RLHF quietly breaks that, and how to actually measure the gap.
- 09 Channel Capacity and the Noisy Channel Shannon's capacity theorem, why it says reliable communication is possible at any rate below capacity and impossible above it, and why the noisy-channel decomposition keeps reappearing in language modelling.
- 10 Kolmogorov Complexity and MDL The shortest program that outputs a string is the ultimate measure of its information content, why it is uncomputable, and how minimum description length turns that uncomputable ideal into a usable model selection principle.
- 11 Language Modelling as Compression Every autoregressive language model is, exactly and not metaphorically, a lossless compressor; arithmetic coding is the bridge, and that equivalence is what let a text-only LLM out-compress PNG and FLAC on images and audio.
- 12 Mutual Information and Representations The information-theoretic quantity that measures how much one variable tells you about another, why it is the right lens for judging whether a learned representation actually captured something, and why estimating it in high dimensions is notoriously hard.
- 13 Rate-Distortion Theory The theory of how few bits a source needs when perfect reconstruction is not required, why every compression decision in an LLM stack is a point on one curve, and what the curve's shape tells you.
- 14 The Softmax Bottleneck Why a standard softmax output layer is a low-rank approximation to the true distribution of language, the "bottleneck" that caps what any single softmax can express, and the mixture trick that breaks it.
- 15 Typical Sets and the AEP Why almost all probability mass sits on a vanishingly small set of sequences that are individually unremarkable, and why the most likely sequence is usually not a typical one.
- 16 f-Divergences Beyond KL KL is one member of a family generated by a convex function, the choice of member decides whether your model covers the data or collapses onto a mode, and some tasks need a divergence that is not in the family at all.
02
Transformer Internals
Open the box. How a language model actually turns text into predictions.
8tracks
116concepts
929cards
14.6hreading
Tokenisation BPE, WordPiece, Unigram, and the ways subword vocabularies quietly shape model behaviour. 15 concepts · 136 cards
- 01 Special and Control Tokens The handful of reserved vocabulary entries no amount of BPE merging could ever produce from ordinary text, and why keeping them outside the mergeable vocabulary is a real security boundary, not just bookkeeping.
- 02 Tokenisation and BPE How models turn raw text into integer tokens, and why the vocabulary choice silently shapes model behaviour.
- 03 Why LLMs Cannot Spell or Count Letters The classic "how many r's in strawberry" failure is not a reasoning gap, it is a representational one, caused by the model never seeing the individual letters that make up a compressed token.
- 04 Byte-Level BPE Why GPT-2 and its descendants tokenise raw bytes instead of characters, and how that single design choice guarantees any input string can be represented without an unknown-token fallback.
- 05 Detokenisation and Streaming Boundaries Streaming a response token by token can briefly render a garbled glyph on screen, a direct, visible consequence of a multi-byte character being split across more than one token, and a correctness problem, not just a UI quirk.
- 06 The Tokenisation Tax Two prompts with the same meaning can cost a different number of tokens under the same vocabulary, and that gap compounds into real dollars, real context budget, and real latency before the model reasons about anything at all.
- 07 Token Fertility and Multilingual Fairness The same paragraph of news text can tokenise into several times as many tokens in one language as another under a shared vocabulary, a structural cost baked in before any user sends a request, not a rounding error.
- 08 Token Healing and Boundary Bias Why a prompt that ends mid-word makes a model complete badly, how greedy tokenisation creates an off-distribution prefix, and what backing up one token fixes.
- 09 Tokenisation and Arithmetic A model can be strong at multi-step reasoning and still botch four-digit addition, not from a lack of arithmetic ability but because its tokeniser handed it two structurally unrelated token sequences for two numbers that differ by one.
- 10 Unigram LM and SentencePiece A tokeniser that builds its vocabulary top-down instead of bottom-up, assigns every segmentation of a string a real probability, and the library that made it (and BPE) usable without a language-specific pre-tokeniser.
- 11 WordPiece Tokenisation BERT's tokeniser looks like BPE on the surface, but the merge criterion it optimises is different, and that difference is why WordPiece pieces tend to track real morphemes more closely.
- 12 BPE-Dropout and Subword Regularisation A deterministic tokeniser has a blind spot no amount of extra training data fixes on its own, the model never has to be robust to an unfamiliar segmentation of a familiar word, until two techniques deliberately introduced randomness into tokenisation as a training-time fix.
- 13 Scaling Laws with Vocabulary Vocabulary size is a scaling parameter like width and depth, most models pick it too small, and the compute-optimal value grows with model size but more slowly than the parameter count.
- 14 Tokeniser Transplantation How to give a pretrained model a different tokeniser without retraining it, why the embedding matrix is the only thing that must change, and what the four families of methods cost.
- 15 Under-Trained and Glitch Tokens Why some tokens in a model's vocabulary were almost never seen during training, what happens when a user types one, and how to find them from the weights alone.
Embeddings & Representations The lookup table, the residual stream, contextual vectors, geometry and superposition. 17 concepts · 177 cards
- 01 Cosine Similarity vs Dot Product Two near-identical looking formulas that answer different questions, one measures direction alone, the other measures direction and magnitude together, and picking the wrong one silently breaks a search or ranking system.
- 02 Embeddings and Semantic Search How dense vectors turn text into a geometry of meaning, and how cosine similarity lets you find related content without keywords.
- 03 Input Embeddings and the Lookup Table How a token id becomes a vector, a single row lookup into a trained matrix, and why that matrix is often the largest single block of parameters a small model spends before any real computation happens.
- 04 word2vec and GloVe The two ideas, predict-a-neighbour and factor-a-co-occurrence-matrix, that first proved dense vectors could capture enough word meaning to support arithmetic on it, years before transformers existed.
- 05 Binary and Int8 Embedding Quantisation Storing each embedding dimension as one bit instead of thirty-two cuts index memory by 32x and, with a float rescoring pass over the shortlist, gives most of the retrieval quality back.
- 06 Embedding Benchmarks and the Zero-Shot Problem The leaderboard that everyone uses to pick an embedding model publishes training splits for its own test sets, so a top rank increasingly measures in-domain fit rather than the out-of-domain generalisation retrieval actually needs.
- 07 Matryoshka Representation Learning Training an embedding model so that the first m dimensions of every vector are themselves a usable embedding, letting one index be read at 64, 256 or 3072 dimensions without re-encoding the corpus.
- 08 Sentence Embeddings and Pooling A transformer produces one vector per token, not one per sentence, so turning a sequence of contextual vectors into a single comparable vector requires a pooling choice that quietly changes what "similar" ends up meaning.
- 09 Static vs Contextual Embeddings The shift from one vector per word to one vector per word-in-context, and why that shift is arguably the single biggest reason transformer-based models outperformed earlier NLP.
- 10 Weight Tying of Input and Output Embeddings Why most language models score every candidate next token using the transpose of the exact same matrix that looked up the input tokens, cutting parameters and improving perplexity in one move.
- 11 Cross-Lingual Embedding Alignment Putting a hundred languages into one vector space so that a Hindi query retrieves an English document requires an explicit alignment signal, and the way you supply it determines exactly how the space fails.
- 12 Embedding Geometry and Anisotropy Learned embedding spaces are not the well-spread sphere the geometric intuition suggests, they collapse into a narrow cone, and that fact quietly breaks naive similarity comparisons built on top of them.
- 13 Hard Negative Mining and Contrastive Embedding Training A retrieval embedder is only as good as the negatives it was trained against, and the gap between easy in-batch negatives and mined hard negatives is the single largest lever in dual-encoder training.
- 14 Hubness in High-Dimensional Retrieval In high-dimensional spaces a small number of points appear in almost everyone's nearest-neighbour list regardless of relevance, which is a property of the geometry rather than a bug in the embedder.
- 15 Superposition and Polysemantic Neurons Individual neurons in a trained network routinely fire for several unrelated concepts at once, and the leading explanation is not noise, it is a model deliberately packing more features than it has dimensions using near-orthogonal directions.
- 16 The Residual Stream Reframing a transformer's residual connections as one shared, additive vector space that every layer reads from and writes to, the lens that makes attention, MLPs, and interpretability results legible as a single system.
- 17 Unembedding and the Logit Lens Multiplying an intermediate layer's residual stream by the final unembedding matrix, as if it were the last layer, turns out to produce surprisingly sensible next-token guesses, a cheap window into what a model has committed to mid-computation.
Attention Internals Queries, keys and values, masking, multi-head and grouped-query, sinks, and the quadratic wall. 14 concepts · 94 cards
- 01 Attention as Soft Dictionary Lookup The mental model that makes attention click before the matrix algebra does - a lookup table where the key match is a matter of degree, not an exact hit or miss.
- 02 Attention Mechanism How attention lets a model focus on the relevant parts of a sequence by computing weighted dependencies between every pair of positions.
- 03 Causal Masking How a transformer trained to predict every position in parallel is stopped from cheating by looking ahead, and why the mask is applied before softmax rather than after.
- 04 Multi-Head Attention in Depth Why attention runs as several small parallel attentions rather than one large one, what each head actually gets to specialise in, and why most heads turn out to be redundant.
- 05 Queries, Keys, and Values The retrieval metaphor behind attention, made literal - what a query, key, and value actually are as learned projections, and why three separate matrices instead of one.
- 06 Self-Attention vs Cross-Attention The one-line distinction, where the queries come from versus where the keys and values come from, and why decoder-only LLMs mostly made cross-attention disappear from the mainstream architecture.
- 07 Sliding-Window Attention The simplest fix to attention's quadratic cost - cap how far back each position can look - and why stacking layers gets that fixed window back to an effectively long range.
- 08 The Quadratic Cost of Attention Where the O(n squared) in "attention is quadratic" actually comes from, why it hits both compute and memory, and what doubling the context length really costs.
- 09 Attention Sinks Why the first few tokens of almost any sequence soak up a disproportionate share of attention regardless of their content, and how that quirk becomes the key to stable infinite-length streaming generation.
- 10 Induction Heads A specific, mechanistically understood attention circuit that copies patterns it has seen once in the current context, and the closest thing the field has to a concrete explanation for in-context learning.
- 11 Linear Attention and Kernel Feature Maps Drop the softmax, replace the exponential kernel with a feature map, and matrix associativity turns quadratic attention into a linear recurrence with constant-size state — plus the quality gap that kept it out of frontier models for five years.
- 12 Multi-Head Latent Attention DeepSeek's answer to the KV cache problem — cache one low-rank latent vector per token instead of every head's keys and values, absorb the up-projection into the query and output weights, and carve out a separate untouched channel for RoPE.
- 13 Multi-Query and Grouped-Query Attention Why the key and value projections, not the query projection, were the ones cut down to fix the real inference bottleneck, and the middle-ground design that most production LLMs settled on.
- 14 Natively Trainable Sparse Attention Most sparse attention is bolted on at inference to a densely trained model, which leaves accuracy on the table and speed unrealised; training sparsity in from the start requires the sparsity pattern to be differentiable and the memory access pattern to suit a GPU.
Positional Encoding Sinusoidal, learned, RoPE, ALiBi, and how context windows get stretched past training length. 16 concepts · 157 cards
- 01 Learned Absolute Positions The simplest positional scheme, a trainable embedding table indexed by slot number, and the hard ceiling it builds into every model that uses it.
- 02 Why Attention Needs Positions Self-attention is a permutation-equivariant set operation by default; every notion of word order a transformer has was injected as data, not built into the architecture.
- 03 ALiBi: Attention with Linear Biases A zero-parameter alternative to rotating or embedding position, a per-head linear penalty on distance, and why train-short-test-long comes at the cost of sharp long-range recall.
- 04 Effective vs Nominal Context Length The advertised context window and the length a model actually uses well are different numbers, often by a large factor, and the gap between them has several distinct, measurable causes.
- 05 Multidimensional RoPE for Images and Video A video is three coordinates, not one, and flattening it into a token index throws away the fact that two patches were vertically adjacent; M-RoPE and 2D RoPE split the head dimension into independent axes so a single mechanism can encode time, height, and width.
- 06 Positional Encodings Why attention needs to be told where each token sits, and how RoPE, ALiBi, and the surprising NoPE result shape how far a model can read.
- 07 Relative Position Representations The 2018 idea that attention should see i-j directly rather than infer it from two absolute positions, the direct ancestor of both RoPE and T5's relative bias.
- 08 Rotary Position Embeddings (RoPE) The rotation trick behind Llama, Mistral, and Qwen, worked through in full, and why an exact algebraic guarantee beats hoping the network learns relative distance on its own.
- 09 Sinusoidal Positional Encodings The original transformer's fix for order, a fixed sin/cos signal added to every embedding, and why its elegant closed form still degrades once you run past training length.
- 10 Contextual Position Encoding Every standard position scheme counts tokens, which is why a model cannot reliably attend to "the previous sentence"; CoPE makes the position counter itself a function of content, incrementing only on tokens the model decides matter.
- 11 Interleaving RoPE and NoPE Layers Llama 4 and Cohere's Command A both build long-context models by using rotary embeddings on most layers and no positional encoding at all on the rest, turning a choice everyone treated as global into a per-layer one.
- 12 Length Extrapolation What it actually means for a model to handle sequences longer than training, why perplexity is a necessary but misleading metric for it, and how different positional schemes fare with zero adaptation.
- 13 Position-Augmented Training If a model fails at length because it has never seen large position indices, the cheapest fix is not a longer training sequence but a shuffled or skipped one, showing the model position 90,000 inside a 2,048-token batch.
- 14 RoPE Scaling: NTK-aware and YaRN How a 4k-token base checkpoint becomes a 128k-token release without pretraining from scratch, by rescaling RoPE's rotation frequencies rather than its raw positions.
- 15 The NoPE Result Decoder-only transformers trained with zero positional encoding of any kind matched or beat RoPE and ALiBi on length generalisation, because the causal mask alone leaks a position signal.
- 16 The RoPE Base Frequency One hyperparameter, written as 10000 in the original RoFormer code and 500000 in Llama 3, sets the entire wavelength spectrum of rotary position embedding and puts a hard ceiling on the context length the model can actually discriminate.
Transformer Anatomy The block, the stack, encoder vs decoder, MoE, and the design choices that separate model families. 14 concepts · 58 cards
- 01 Anatomy of a Transformer Block The exact sequence of operations inside one transformer block, from tensor shapes to parameter counts, and why every frontier model is just this same function stacked dozens of times.
- 02 Decoder-Only and Why It Won Why a single causal stack with one unified training objective beat two-stack and bidirectional designs at scale, and the bidirectional context it deliberately gives up to get there.
- 03 Depth, Width, and Aspect Ratio Given a fixed parameter budget, why pretraining loss barely cares whether you go deep or wide, and the hardware and capability reasons architects still don't pick the ratio at random.
- 04 Encoder-Decoder Models (T5) T5's text-to-text framing and the extra cross-attention sublayer that lets a decoder condition on a separately-encoded input, and the real cost/benefit case for keeping two stacks instead of one.
- 05 Encoder-Only Models (BERT) BERT's bidirectional masked-language-model objective, why it makes phenomenal embeddings and classifiers, and why that same bidirectionality makes it structurally unable to generate open-ended text.
- 06 Feed-Forward Networks and SwiGLU The other half of every transformer block, where most of the parameters and arguably most of the stored knowledge live, and why the activation function quietly became SwiGLU.
- 07 Layer Normalization and Residual Connections The two pieces of transformer plumbing that make deep stacks trainable at all, why pre-norm beat post-norm, and how RMSNorm shaved the design down further.
- 08 Parallel Attention and FFN Computing attention and the feed-forward sublayer from the same normalised input instead of sequentially, trading a small representational restriction for less synchronisation on the way to more throughput at scale.
- 09 Sub-Layer Ordering and Design Choices The menu of ordering decisions inside and across transformer blocks beyond pre-norm versus post-norm, from where the final normalisation sits to why attention always runs before the feed-forward sublayer.
- 10 Transformer Architecture The encoder-decoder stack that replaced recurrence and powered every modern LLM.
- 11 Mixture of Experts Why MoE models can be 10x cheaper to serve than dense models of the same capability, and what makes them hard to train.
- 12 Prefix Language Models A single decoder-only stack that grants bidirectional attention to a prefix segment before switching to causal generation, and why the field mostly passed on this compromise anyway.
- 13 QK-Normalisation Normalising queries and keys before the attention dot product to stop logits from blowing up at scale, the fix that made 22-billion-parameter and larger transformers trainable without loss spikes.
- 14 Weight Initialisation in Transformers Why the last matrix in every attention and FFN sublayer gets shrunk by 1/sqrt(2N) at initialisation, and the more principled framework, muP, that turns hyperparameter tuning at scale into a lookup instead of a search.
Training Objectives Next-token prediction, masked LM, span corruption, fill-in-the-middle, and auxiliary losses. 14 concepts · 76 cards
- 01 Next-Token Prediction and Cross-Entropy Loss The single objective behind every LLM - predict the next token - and the cross-entropy loss that turns "predict well" into a gradient the model can descend.
- 02 Softmax and Logits How a model's raw output scores become a probability distribution, why the exponential matters, and the numerical trick that keeps softmax from overflowing.
- 03 The Causal LM Pretraining Task How a corpus of raw documents becomes millions of training examples under one masking rule, and why "causal" describes a data and attention decision, not just a loss.
- 04 Fill-in-the-Middle A way to teach a plain causal decoder to infill text by rearranging training documents, not the model, so a single architecture serves both left-to-right generation and code-editor-style completion.
- 05 Label Smoothing Softening a one-hot target so the model is never rewarded for driving a correct-class probability all the way to 1, trading a little training loss for calibration and generalisation.
- 06 Loss Masking in Fine-Tuning Supervised fine-tuning uses the same shifted next-token loss as pretraining, computed over the exact same kind of sequence; the only change is that most of the sequence is excluded from the loss entirely.
- 07 Masked Language Modelling BERT's pretraining task hides tokens instead of futures, trading the ability to generate text for representations that read both directions at once.
- 08 Pretraining Objectives Why the loss a model is trained to minimise decides what it can become, and how next-token prediction beat masked and span objectives to own the generative era.
- 09 Sequence Packing and Document Masking Concatenating short documents to fill a fixed context length eliminates padding waste, but only if the attention mask and position IDs are made document-aware; done naively, packing quietly teaches the model to attend across unrelated documents.
- 10 Span Corruption T5's pretraining task hides whole spans instead of single tokens and asks a decoder to generate only the missing pieces, trading BERT's per-token reconstruction for a shorter, cheaper target sequence.
- 11 MoE Load-Balancing Loss Without an explicit penalty, a mixture-of-experts router collapses onto a handful of favourite experts within the first few hundred steps; the auxiliary load-balancing loss is the mechanism that stops it.
- 12 Multi-Token Prediction Training a language model to predict the next n tokens at once from a shared trunk, which densifies the training signal and hands you free draft heads for self-speculative decoding.
- 13 The Softmax Cross-Entropy Gradient The combined derivative of softmax and cross-entropy collapses to one subtraction, predicted probability minus true label, and that simplification is why every production training loop fuses the two ops instead of computing them separately.
- 14 Z-Loss and Logit Regularisation Softmax is shift-invariant, which leaves a direction in logit space that cross-entropy never penalises; z-loss closes that gap and is what keeps large-scale bf16 training from spiking.
Decoding & Generation Greedy, beam, temperature, top-k, nucleus, and constrained generation into structured formats. 16 concepts · 143 cards
- 01 Greedy Decoding The simplest possible decoder, take the single most probable token at every step, and why that local optimality guarantees nothing about the sentence you end up with.
- 02 Temperature Sampling Dividing logits by a scalar before softmax to sharpen or flatten a model's output distribution, and why temperature alone is a poor substitute for a real safety mechanism.
- 03 Top-k Sampling Truncating the vocabulary to a fixed-size shortlist of the k most probable tokens before sampling, and why a fixed k is systematically the wrong size for at least some fraction of every distribution.
- 04 Beam Search Keeping the k best partial sequences alive instead of committing to one token at a time, and why the strategy that dominates machine translation actively hurts open-ended generation.
- 05 Determinism and Reproducibility in Decoding Why "temperature zero" and "same seed" both promise less determinism than they sound like they do, and where the real sources of nondeterminism live in an inference stack.
- 06 Entropy-Adaptive Sampling A fixed top-k or top-p threshold applies the same truncation to a distribution with one plausible continuation and to one with two hundred; adaptive samplers set the cut from the shape of the distribution itself.
- 07 Logit Bias and Masking Directly editing the logit vector before sampling, additive nudges for soft steering, hard -infinity masks for guaranteed exclusion, and why the two are not interchangeable.
- 08 Neural Text Degeneration The umbrella diagnosis behind most decoding research, why decoders that maximise sequence probability produce measurably worse text than decoders that sample from the model's actual distribution.
- 09 Nucleus (Top-p) Sampling Truncating by cumulative probability mass instead of a fixed rank, so the candidate pool automatically shrinks when the model is confident and grows when it is uncertain.
- 10 Repetition Penalty and No-Repeat N-Grams Two blunt but effective ways to stop a decoder from looping, subtracting from the logits of tokens already seen, versus hard-banning any n-gram that has already appeared.
- 11 Sampling and Decoding A language model outputs a probability distribution, not text; the decoding strategy turns that distribution into words and quietly decides whether the output is dull, unhinged, or right.
- 12 Stopping Criteria and EOS Calibration Generation ends when the model emits an end-of-sequence token, when a stop string matches, or when the token budget runs out, and the three failure modes look identical from outside while having completely different causes.
- 13 Structured Generation and Constrained Decoding How masking the logits at each decode step to only tokens a schema or grammar allows guarantees syntactically valid output, and where that guarantee stops.
- 14 Contrastive Decoding and DoLa If a small model's failures are an exaggerated version of a large model's failures, the difference between their logits is a usable quality signal, and the same trick works between the early and late layers of a single model.
- 15 Min-p and Typical Sampling Two later refinements to nucleus sampling, one that scales the truncation threshold to the top token's own confidence, one that truncates by information content rather than raw probability rank.
- 16 Minimum Bayes Risk Decoding Every standard decoder searches for the most probable sequence, and a decade of machine translation research shows that the mode of a neural sequence model is frequently degenerate; MBR replaces maximisation with expected-utility estimation.
Context & In-Context Learning Autoregressive generation, the prompt stack, context engineering, and long-context degradation. 10 concepts · 88 cards
- 01 Autoregressive Generation How a language model turns next-token prediction into a paragraph, and why generation is a loop that feeds its own output back as input.
- 02 Context Compaction and Handoff When an agent's conversation approaches the window limit, compaction summarises the history and reinitialises a fresh window from the summary; what survives the compression determines whether the agent continues or silently restarts.
- 03 Context Engineering The discipline of curating exactly which tokens occupy the model's window during inference, and why it became the core skill for building agents that run longer than a single turn.
- 04 In-Context Learning How large models learn a task from examples in the prompt alone, with no weight updates, and why this emergent ability reframed how we use LLMs.
- 05 Many-Shot In-Context Learning What changes when you put hundreds or thousands of examples in the prompt instead of five, why the gains keep coming after few-shot plateaus, and how model-generated rationales substitute for scarce human data.
- 06 RAG vs Long Context The engineering decision the million-token window forced, what controlled comparisons actually found about quality and cost, and why routing between retrieval and full-context beats picking a side.
- 07 The Prompt Stack and Chat Roles What a chat prompt actually is under the hood: a single token sequence built from system, user, and assistant turns wrapped in special tokens, and why that structure is load-bearing.
- 08 Context Rot The measured fact that model accuracy falls as the input grows, non-uniformly and in cliffs, so a bigger window is a bigger desk rather than a better memory.
- 09 Context Windows and Long-Context Models Why a model advertised at a million tokens can still lose the fact in the middle, and what actually sets the limit: memory, compute, position, and attention itself.
- 10 Long-Context Training Recipes How a model trained at 8k becomes a genuine 128k model, why the data mixture matters more than the token count, and what the Llama 3 and ProLong recipes agree on.
03
Training & Fine-Tuning
From raw web crawl to an aligned model — data, dynamics, scale and adaptation.
6tracks
110concepts
886cards
14.2hreading
Pretraining Data Pipelines Web-scale corpus construction, filtering, deduplication, decontamination and data mixtures. 20 concepts · 140 cards
- 01 Benchmark Decontamination Benchmark decontamination is the process of identifying and removing evaluation set examples from a model's pretraining corpus so that reported benchmark scores reflect genuine generalisation rather than memorised answers.
- 02 Curriculum and Data Ordering How the sequence and mixture proportions of training batches affect what an LLM learns and when, and why naive i.i.d. sampling often leaves capability on the table.
- 03 Data Provenance and Licensing Data provenance tracks the origin and transformation history of every byte in a training corpus, while licensing determines whether you are legally permitted to use it at all.
- 04 Deduplication and Memorisation Duplicate training examples cause language models to memorise and verbatim-reproduce training text, so removing duplicates before training reduces regurgitation, improves generalisation, and lowers compute waste.
- 05 Document-Level vs Token-Level Deduplication Document-level deduplication removes whole near-duplicate pages using hashing, while token-level deduplication removes repeated spans within and across documents using suffix arrays, each with distinct cost-quality trade-offs in web-scale corpus construction.
- 06 Exact and Fuzzy Deduplication How hash-based exact matching and MinHash locality-sensitive hashing together remove the duplicate content that otherwise inflates memorisation and wastes training compute.
- 07 Heuristic Quality Filters Heuristic quality filters are rule-based passes over raw web text that remove boilerplate, malformed documents, and low-information content before any classifier is trained or applied.
- 08 Language Identification and Filtering Language identification assigns a language label to each document in a web crawl so pipelines can retain target-language text and discard everything else before quality filtering begins.
- 09 MinHash and LSH for Deduplication MinHash estimates Jaccard similarity between documents in constant memory, and LSH bucketing turns that estimate into a sub-linear nearest-neighbour search, making corpus-scale near-deduplication tractable.
- 10 Multilingual Data Balancing Multilingual data balancing decides how many tokens from each language land in a pretraining corpus, trading raw web proportions against deliberate upsampling to serve low-resource languages without degrading high-resource ones.
- 11 PII Detection and Removal Scrubbing personally identifiable information from web-scale corpora before LLM pretraining reduces memorisation risk and legal exposure, but every detection method trades recall against corpus damage.
- 12 Quality Filtering with Classifiers Classifier-based quality filtering uses lightweight models trained on curated reference corpora to score and discard low-quality web documents before LLM pretraining.
- 13 Text Extraction and Boilerplate Removal Converting raw HTML from web crawls into clean, main-content text is a lossy signal-recovery problem, and the choices made here propagate irreversibly through every downstream filtering and training stage.
- 14 Toxicity Filtering of Pretraining Data Toxicity filtering removes hate speech, obscene language, and harmful content from web-crawled corpora before LLM pretraining, using blocklists, classifiers, and API-based scoring, each with distinct recall/precision trade-offs and equity side effects.
- 15 Training a Tokeniser on a Corpus Training a tokeniser on your pretraining corpus, rather than borrowing one designed for another model, directly controls sequence length, vocabulary coverage, and how the model sees numbers, code, and non-English text.
- 16 Vocabulary Size Trade-offs Choosing a tokeniser vocabulary size forces a three-way tension between sequence length, embedding table memory, and coverage of rare or multilingual text.
- 17 Code Data Curation A systematic account of how raw source code from the internet is transformed into a deduplicated, filtered, mixed, and decontaminated pretraining corpus for code-focused language models.
- 18 Data Mixtures and Domain Weighting Domain weighting determines how much of each data source a model sees during pretraining, and getting this wrong can cost tens of thousands of GPU-hours or silently cripple downstream task performance.
- 19 Data Pipelines at Scale Building a pretraining corpus requires extracting, filtering, deduplicating, and mixing hundreds of billions of tokens from heterogeneous sources while keeping benchmark contamination out.
- 20 Web-Scale Corpus Construction Building a pretraining corpus at web scale requires five tightly coupled stages - extraction, quality filtering, deduplication, source mixing, and decontamination - each of which silently determines what a model can and cannot know.
Synthetic Data Self-Instruct, distillation, self-play, and how to generate training data without collapsing. 20 concepts · 140 cards
- 01 Why Synthetic Data Synthetic data lets practitioners generate arbitrarily large, precisely labelled training sets without manual annotation, but only when the generator is accurate enough not to amplify its own errors.
- 02 Auditing Synthetic Data Auditing synthetic training data means applying measurable quality checks, coverage tests, and contamination scans before the data ever touches a training run.
- 03 Distillation and Terms-of-Service Constraints Using a commercial API to generate training data for a competing model almost universally violates the provider's terms of service, and understanding exactly why - and what compliant alternatives exist - is non-negotiable before building any distillation pipeline.
- 04 Distilling Reasoning Traces Reasoning-trace distillation transfers step-by-step chain-of-thought outputs from a large teacher model into fine-tuning data for a smaller student, giving the student reasoning capability it could not develop on final-answer supervision alone.
- 05 Diversity Metrics and Collapse Detection Quantitative metrics for detecting when synthetic training data loses coverage of the original distribution, with practical monitoring strategies to catch collapse before it harms the next model generation.
- 06 Evol-Instruct Evol-Instruct is an LLM-driven pipeline that iteratively rewrites seed instructions into progressively harder variants, enabling automated construction of complex instruction-following training data without human labellers.
- 07 Knowledge Distillation from a Teacher Model Knowledge distillation trains a smaller student model to reproduce a larger teacher's output distribution, enabling compact models with performance well beyond what their size alone would predict.
- 08 Mixing Synthetic and Human Data Balancing synthetic and human-authored training examples determines whether a model inherits the strengths of both sources or the worst of each.
- 09 Persona Prompting for Diversity Persona prompting injects fictional user identities into an LLM's prompt to steer it toward generating training data that spans a broader slice of the real distribution than naive repeated sampling achieves.
- 10 Programmatic and Templated Data Generation Programmatic and templated generation produces synthetic training data through deterministic code, slot-filling templates, and context-free grammars, giving precise control over distribution and format that purely model-driven pipelines cannot match.
- 11 Quality Filtering of Synthetic Data Quality filtering removes low-signal, redundant, or contaminated examples from synthetically generated datasets before fine-tuning, preventing reward hacking and model collapse.
- 12 Rejection Sampling Fine-Tuning and STaR Rejection sampling fine-tuning and the STaR algorithm are iterative self-improvement techniques that generate synthetic training data by having a model solve problems, keeping only the correct solutions, and re-training on those filtered traces.
- 13 Self-Instruct Self-Instruct is a bootstrapping pipeline that uses a language model's own outputs to generate large-scale instruction-following training data with minimal human annotation.
- 14 Self-Play and Self-Improvement Self-play loops use a model to generate, critique, and filter its own training data, compounding capability without proportionally scaling human annotation effort.
- 15 Synthetic Data for Code Generating synthetic code training data via instruction synthesis, distillation, and execution-based filtering lets small models punch well above their weight, but only when a reliable verifier anchors the loop.
- 16 Synthetic Data for Mathematics Synthetic math data pipelines use teacher models, rejection sampling, and question rewriting to bootstrap training corpora far larger than any human-labelled set, but the quality ceiling is set by the generator's own reasoning ability.
- 17 Synthetic Preference Data Synthetic preference data replaces expensive human comparison labels by having a language model judge which of two candidate responses is better, enabling scalable RLHF-style alignment without a large human labelling workforce.
- 18 Textbook-Quality Synthetic Data How to generate training data that teaches models to reason rather than memorise, using instruction synthesis, distillation, rejection sampling, and constitutional loops, along with the collapse risks that follow.
- 19 Model Collapse from Recursive Training When a model is trained repeatedly on its own outputs, tail distributions erode and the model progressively forgets rare but important knowledge, eventually producing impoverished, homogenised text.
- 20 The Constitutional AI Data Loop Constitutional AI replaces most human preference labels with a self-critique-and-revise loop guided by a written list of principles, producing both supervised fine-tuning data and AI-labelled preference pairs that train a reward model.
Training Dynamics & Scaling Learning-rate schedules, warmup, loss spikes, critical batch size, muP, and scaling laws. 16 concepts · 157 cards
- 01 Gradient Accumulation The trick that lets a single GPU simulate a batch size far larger than what fits in its memory, at the cost of wall-clock time rather than compute.
- 02 Gradient Clipping A five-character config value, clip norm 1.0, appears in nearly every published pretraining recipe, and it is a crude safety net rather than a fix, one that is easy to misread as more protective than it is.
- 03 Learning-Rate Schedules The shape of the learning-rate curve across a training run matters as much as its peak value, and getting the shape wrong wastes a slice of a compute budget that was never coming back.
- 04 Optimiser State and Memory A model's raw weight size is only the starting point for training memory; Adam alone roughly quadruples the footprint before a single activation has been stored.
- 05 Warmup and Why It Helps The first few hundred to few thousand steps of a large training run are its most fragile, and a short linear ramp on the learning rate is the cheapest insurance against wrecking them.
- 06 Weight Decay in Pretraining Pretraining rarely revisits the same token twice, so the classical overfitting story for weight decay barely applies, yet nearly every LLM recipe still sets it to a nonzero value close to 0.1.
- 07 Double Descent Test error rises to a peak at the interpolation threshold and then falls again as models grow past it, which breaks the U-shaped bias-variance picture and explains why more parameters can be safe.
- 08 Emergent Abilities and Metric Artefacts Capabilities that appear to switch on abruptly at a critical scale, the argument that the abruptness comes from the metric rather than the model, and what survives the critique.
- 09 Grokking and Delayed Generalisation Networks that memorise a small dataset perfectly and then, thousands of steps later, suddenly generalise, why the delay happens, and what the four competing explanations agree on.
- 10 Loss Spikes and Divergence On almost every long pretraining run the smooth descending loss curve suddenly lurches upward, and diagnosing and recovering from these spikes is still closer to an operational skill than a solved engineering problem.
- 11 Scaling Laws and the Chinchilla Correction How loss falls predictably with compute, parameters, and data, and why the Chinchilla result showed almost every large model of its era was badly undertrained.
- 12 The Critical Batch Size Past a point that can be measured but not derived from first principles, adding more GPUs to widen the batch stops buying faster convergence per token, and just burns compute for redundant gradient information.
- 13 The Edge of Stability Gradient descent on neural networks does not stay in the regime where its convergence theory applies; the loss curvature grows until training is marginally unstable, and then hovers there.
- 14 The Neural Tangent Kernel and Feature Learning In the infinite-width limit a network trains like a fixed kernel machine, which makes it analysable and also strictly weaker than real networks, and the gap between the two regimes is where deep learning's advantage lives.
- 15 Warmup-Stable-Decay Schedules Cosine decay bakes the total step count into its formula before training starts; warmup-stable-decay schedules split the learning-rate curve so that commitment can be made at the very end instead.
- 16 muP and Hyperparameter Transfer The learning rate that is optimal for a 40-million-parameter proxy model is usually wrong for the 70-billion-parameter model it was meant to stand in for, and muP is the parametrisation designed to make that transfer actually work.
Distributed Training Data, tensor and pipeline parallelism, ZeRO/FSDP sharding, mixed precision, and offload. 13 concepts · 165 cards
- 01 Data Parallelism and DDP How replicating the model and sharding the batch across GPUs scales training, and why AllReduce is the primitive every framework eventually depends on.
- 02 Fault Tolerance at Scale Why a 16,000-GPU run fails every few hours, how checkpoint interval trades wasted compute against write cost, and what asynchronous and in-memory checkpointing changed.
- 03 Gradient Checkpointing, Activation Recomputation, and CPU Offload Why activations - not weights - usually dominate training memory, and how recomputation and CPU/NVMe offload trade compute and bandwidth to fit larger models.
- 04 Mixed-Precision Training (FP16, BF16, FP8) How lower-precision formats halve memory and double throughput on tensor cores, why BF16 displaced FP16 for training, and what FP8 changes on H100 and Blackwell.
- 05 Composing Parallelism Strategies How data, tensor, pipeline, context and expert parallelism combine into one device mesh, why the ordering follows the interconnect hierarchy, and what each axis costs.
- 06 Expert Parallelism and All-to-All Sparse mixture-of-experts models are sharded by placing different experts on different devices, which replaces the all-reduce of dense training with two all-to-all exchanges per layer and makes routing a network-topology problem.
- 07 Gradient Compression and Quantised Collectives Ninety-nine point nine percent of the values in a distributed SGD gradient exchange are redundant, and twenty years of compression research shows that exploiting this is easy in theory and constrained in practice by one property: whether the compressed form survives an all-reduce.
- 08 Low-Communication Distributed Training Standard data parallelism synchronises gradients every step, which requires a datacentre-grade fabric; DiLoCo synchronises every few hundred steps instead and trains language models across poorly connected islands of accelerators.
- 09 Overlapping Communication and Computation Why a large training run's collectives are mostly free when overlapped and catastrophic when exposed, and the bucketing, prefetch and scheduling tricks that hide them.
- 10 Pipeline Bubbles and Schedules Why pipeline parallelism wastes device time by construction, how the bubble fraction depends on micro-batch count, and what 1F1B, interleaving and zero-bubble schedules recover.
- 11 Sequence and Context Parallelism Why long-context training runs out of memory even when the model fits, and how Ring Attention and Ulysses split the sequence dimension across devices without changing the maths.
- 12 Tensor and Pipeline Parallelism How frontier labs split a model across thousands of GPUs by sharding within layers (tensor parallel) and across layers (pipeline parallel), and how to pick the split.
- 13 ZeRO and FSDP How sharding optimiser state, gradients, and parameters across data-parallel ranks turns a memory problem into a bandwidth problem, and why FSDP is now the PyTorch default.
Parameter-Efficient Fine-Tuning LoRA, QLoRA, DoRA, adapters and soft prompts — adapting big models on small budgets. 20 concepts · 140 cards
- 01 Adapter Modules Adapter modules insert small trainable bottleneck layers into a frozen pretrained transformer, achieving near-full fine-tuning performance while updating fewer than 4% of the model's parameters.
- 02 Catastrophic Forgetting in Fine-Tuning Full fine-tuning on a narrow task silently destroys general capabilities baked in during pre-training, and understanding why this happens is the prerequisite for choosing any mitigation strategy.
- 03 Choosing LoRA Rank and Alpha LoRA rank r controls how much task-specific capacity the adapter has, and alpha controls the scaling of that update; choosing them poorly wastes parameters or destabilises training.
- 04 IA3 and Scaling-Vector Methods IA3 fine-tunes a transformer by learning one scaling vector per targeted activation stream, achieving roughly 0.01% trainable parameters while matching full fine-tuning accuracy on several benchmarks.
- 05 LoRA Training Pitfalls LoRA's low-rank approximation introduces subtle failure modes around rank selection, learning rate asymmetry, and target-module coverage that can silently degrade fine-tuned model quality.
- 06 LoRA vs Full Fine-Tuning LoRA constrains weight updates to low-rank matrices, cutting trainable parameters by orders of magnitude while matching full fine-tuning quality on most tasks, but that constraint is also the source of its failure modes.
- 07 LoRA: Low-Rank Adaptation LoRA freezes a pretrained model's weights and inserts trainable low-rank matrix pairs into each target layer, cutting trainable parameters by up to 10,000x with no added inference latency.
- 08 Merging LoRA into Base Weights After LoRA training, the low-rank adapter matrices can be folded directly into the frozen base weights, eliminating inference overhead and adapter management complexity while producing a standard dense model.
- 09 Prefix Tuning Prefix tuning freezes all pretrained model weights and instead optimises a small set of continuous, task-specific vectors prepended to every layer's key-value cache, achieving within a few points of full fine-tuning while training roughly 0.1% of the original parameters.
- 10 Prompt Tuning with Soft Prompts Soft-prompt tuning prepends a small set of learnable continuous vectors to the input of a frozen language model, achieving full-fine-tuning parity at billion-parameter scale while touching less than 0.1% of model weights.
- 11 QLoRA: 4-bit Base with LoRA QLoRA reduces the GPU memory needed to fine-tune a 65-billion-parameter model from hundreds of gigabytes to a single 48 GB card by storing the frozen base model in 4-bit precision and training only small LoRA adapters at full 16-bit precision.
- 12 The Intrinsic-Dimension Hypothesis Pre-trained language models can be fine-tuned in a surprisingly low-dimensional subspace of their parameter space, and measuring that dimension explains why parameter-efficient methods work at all.
- 13 The Memory Maths of Fine-Tuning Fine-tuning a 7B-parameter model with full gradients and Adam state consumes roughly 112 GB of GPU memory; parameter-efficient methods cut that by an order of magnitude by training only a tiny fraction of weights.
- 14 Which Layers to Adapt Choosing which transformer weight matrices to inject LoRA or adapter modules into determines both the parameter budget and downstream task quality, and the right choice is not self-evident.
- 15 Why Parameter-Efficient Fine-Tuning Parameter-efficient fine-tuning methods adapt large pretrained models to new tasks by training only a small fraction of parameters, making customisation practical without the compute and storage costs of full fine-tuning.
- 16 Composing and Stacking Adapters Multiple trained adapters can be combined sequentially, by weighted sum, or through attention-based gating to build new capabilities without retraining the base model.
- 17 DoRA: Weight-Decomposed LoRA DoRA decomposes pre-trained weights into magnitude and direction components, then applies LoRA exclusively to the directional part, closing most of the accuracy gap between LoRA and full fine-tuning without adding inference overhead.
- 18 LoRA for Long-Context Adaptation Extending a model's context window via LoRA requires coordinating low-rank weight updates with position-encoding rescaling, and ignoring either side reliably degrades performance on long sequences.
- 19 NF4 and Double Quantisation NF4 is a 4-bit data type matched to the normal distribution of pretrained weights, and double quantisation further compresses the quantisation constants themselves, together enabling 65B-parameter models to fine-tune on a single 48 GB GPU via QLoRA.
- 20 Serving Many LoRA Adapters How specialised inference systems batch requests across hundreds of distinct LoRA adapters without duplicating the base model weights on GPU.
Alignment & Post-Training SFT, reward modelling, DPO/IPO/KTO/ORPO, model merging, and evaluating an aligned model. 21 concepts · 144 cards
- 01 Building a Preference Dataset A preference dataset pairs model outputs and records which one a human (or AI judge) preferred, providing the training signal that separates a helpful assistant from a raw base model.
- 02 Chat Templates and Special Tokens Chat templates are Jinja2 strings stored in a tokeniser that convert structured message lists into the exact token sequences a fine-tuned model was trained to process.
- 03 DPO in Practice DPO eliminates the separate reward model and RL loop of classic RLHF by reparameterising the reward directly into a classification loss over preferred and rejected response pairs.
- 04 Evaluating an Aligned Model Evaluating an aligned model requires measuring three partially competing properties simultaneously: helpfulness, harmlessness, and honesty, and every method for doing so introduces its own systematic biases.
- 05 KTO: Unpaired Preference Learning KTO aligns language models using only binary good/bad labels per response, avoiding the paired (chosen, rejected) format that makes preference data expensive and brittle to collect.
- 06 Length Bias and Verbosity Control Reward models trained on human preference data systematically score longer outputs higher regardless of quality, causing RLHF-trained models to inflate response length rather than improve content.
- 07 Model Merging: Linear and SLERP Linear and SLERP merging combine the weight tensors of separately fine-tuned models into a single deployable checkpoint, trading off alignment and capability at zero inference cost.
- 08 Model Soups Model soups average the weights of multiple independently fine-tuned checkpoints to produce a single model that outperforms any individual checkpoint without increasing inference cost.
- 09 ORPO and Reference-Free Alignment ORPO collapses supervised fine-tuning and preference alignment into a single training phase by appending a log-odds-ratio penalty directly to the NLL loss, removing the need for a reference model.
- 10 RLAIF and Constitutional Feedback RLAIF replaces human preference labels with an AI judge, and Constitutional AI extends this by encoding a written list of principles so the model critiques and revises its own outputs before any RL training begins.
- 11 Reward Hacking in Alignment Reward hacking occurs when a model maximises its training reward signal through behaviours that violate the designer's intent, undermining alignment despite high measured scores.
- 12 Reward Modelling in Practice A reward model is a learned surrogate for human preference that RLHF uses to provide a differentiable training signal, and its quality determines how aligned the final policy is.
- 13 Supervised Fine-Tuning for Instructions Supervised fine-tuning on curated instruction-response pairs is the first step that transforms a raw pretrained language model into a model that reliably follows human instructions.
- 14 The KL Penalty and Reference Model The KL penalty constrains a fine-tuned language model to stay statistically close to its pre-trained reference, preventing reward hacking while preserving the capabilities built during pretraining.
- 15 The Post-Training Pipeline A structured walkthrough of the four-stage process that converts a raw pretrained language model into a deployable assistant, from supervised fine-tuning through reward modelling, RLHF, and preference-optimisation alternatives.
- 16 DPO and Preference Optimisation How Direct Preference Optimisation collapses the reward-model-plus-PPO pipeline into a single classification loss, and where the RLHF machinery still earns its keep.
- 17 IPO and the Overfitting Fix IPO replaces DPO's sigmoid loss with a squared identity transform, eliminating the theoretical overfitting guarantee that breaks when preference data is finite and deterministic.
- 18 Multi-Objective Alignment Multi-objective alignment trains a single language model to satisfy several competing criteria simultaneously by navigating the Pareto front of reward trade-offs rather than collapsing them into one scalar.
- 19 Online vs Offline Preference Optimisation Offline preference optimisation trains on a fixed dataset of ranked responses, while online methods continuously sample from the current policy, and that single difference has substantial consequences for distribution coverage, reward hacking risk, and final alignment quality.
- 20 PPO for RLHF in Practice A concrete walkthrough of how Proximal Policy Optimisation is wired into the RLHF pipeline, covering the four-model setup, the clipped objective, KL penalty shaping, and the failure modes that kill real training runs.
- 21 TIES and DARE Merging TIES and DARE are two parameter-space merging algorithms that resolve weight interference when combining multiple fine-tuned models into one, avoiding retraining entirely.
04
Reinforcement Learning
Classical RL, then the specific dialect of it that post-trains language models.
5tracks
61concepts
489cards
7.7hreading
RL Foundations MDPs, value functions, TD learning, policy gradients, actor-critic, TRPO and PPO. 20 concepts · 140 cards
- 01 Returns, Discounting, and Episodes The return is the quantity a reinforcement learning agent actually optimises; discounting controls how far into the future it looks, and whether interactions are episodic or continuing shapes which formulation applies.
- 02 Actor-Critic Methods Actor-critic methods combine a policy network (actor) with a value estimator (critic) to reduce variance in policy gradient updates without the high bias of pure value-based methods.
- 03 Baselines and Variance Reduction A baseline is a state-dependent function subtracted from the return in policy gradient updates to reduce estimator variance without introducing bias.
- 04 Deep Q-Networks DQN combines Q-learning with a deep convolutional network and two stabilisation tricks (experience replay and a target network) to learn Atari-level control policies directly from raw pixels.
- 05 Dynamic Programming for RL Dynamic programming solves RL problems exactly by bootstrapping value estimates across states using the Bellman equations, but only when you have a perfect model of the environment.
- 06 Entropy Regularisation Entropy regularisation adds a bonus term to the RL objective that rewards stochastic policies, improving exploration and preventing premature convergence to deterministic optima.
- 07 Markov Decision Processes A Markov Decision Process is the formal framework that turns vague "learn from interaction" intuitions into a precise mathematical problem a computer can solve.
- 08 Monte Carlo Methods Monte Carlo methods estimate value functions by averaging complete episode returns, making them the simplest model-free approach but one that requires episodic tasks and carries high variance.
- 09 On-Policy vs Off-Policy Learning On-policy methods learn from data collected by the policy being updated, while off-policy methods learn from data generated by a different behaviour policy, enabling experience reuse but requiring bias corrections.
- 10 Policy Gradients and REINFORCE Policy gradient methods directly optimise a stochastic policy by estimating the gradient of expected return through sampled trajectories, sidestepping the need to represent a value function over every state-action pair.
- 11 Policy Methods vs Value Methods Policy methods optimise a parameterised policy directly via gradient ascent on expected return, while value methods learn a value function and derive behaviour from it; the distinction shapes sample efficiency, stability, and action-space suitability across the entire RL algorithm landscape.
- 12 Q-Learning Q-learning is a model-free, off-policy temporal-difference algorithm that estimates the value of (state, action) pairs and converges to an optimal policy without requiring a model of the environment.
- 13 Reward Shaping and Credit Assignment Reward shaping injects domain knowledge into the reward signal to speed up learning, while credit assignment determines which past actions actually caused a delayed reward.
- 14 Temporal-Difference Learning TD learning combines the trial-and-error sampling of Monte Carlo with the bootstrapped updates of dynamic programming to learn value functions online, without waiting for episode ends.
- 15 The Exploration-Exploitation Trade-off Choosing when to try something new versus repeating what already works is the central tension in reinforcement learning, and getting it wrong kills agent performance regardless of how well the rest of the system is designed.
- 16 Value Functions and the Bellman Equations Value functions assign expected cumulative reward to states and state-action pairs; the Bellman equations express these values as self-consistent recursive relationships that underpin every practical RL algorithm.
- 17 Generalised Advantage Estimation GAE introduces a single hyperparameter lambda that smoothly interpolates between high-bias/low-variance TD(0) and low-bias/high-variance Monte Carlo advantage estimates, making policy gradient training substantially more stable.
- 18 Model-Based Reinforcement Learning Model-based RL learns an explicit dynamics model of the environment and uses it for planning or synthetic data generation, trading model bias for dramatic gains in sample efficiency.
- 19 Proximal Policy Optimisation PPO stabilises policy gradient training by clipping the probability ratio between old and new policies, preventing destructively large updates without the computational overhead of second-order methods.
- 20 Trust-Region Policy Optimisation TRPO is a policy-gradient algorithm that enforces a KL-divergence constraint on each update, guaranteeing monotonic policy improvement and preventing the catastrophic performance collapses that plague vanilla gradient ascent.
RL for Language Models RLHF as an RL problem, KL-regularised objectives, GRPO, RLVR, and reward over-optimisation. 21 concepts · 142 cards
- 01 Best-of-N and Inference-Time Selection Best-of-N sampling generates multiple completions from a language model and returns the one ranked highest by a reward model, trading inference compute for quality without updating any weights.
- 02 Length and Format Reward Hacking When a reward model assigns higher scores to longer or more structured responses regardless of quality, RL training exploits that signal and the policy degrades into verbose padding and hollow formatting instead of improving reasoning.
- 03 Process vs Outcome Rewards Process reward models score each reasoning step individually, giving denser training signal than outcome rewards and catching errors before they contaminate a final answer.
- 04 RLHF as a Reinforcement-Learning Problem RLHF recasts language model alignment as a policy-optimisation problem where a reward model trained on human comparisons provides the scalar signal that PPO uses to update the language policy.
- 05 Rejection Sampling as RL Rejection sampling fine-tuning filters a model's own outputs by correctness and trains on the survivors, achieving a policy-improvement step that is mathematically equivalent to one round of RL but without an explicit optimiser loop.
- 06 Reward Models as Learned Rewards A reward model is a classifier trained on human preference comparisons that outputs a scalar score, standing in for the true human utility function during RL fine-tuning of language models.
- 07 Reward Over-Optimisation When a language model is trained too aggressively against a proxy reward model, it learns to exploit the proxy rather than genuinely improve, causing measured reward to climb while true quality declines.
- 08 The Bandit Framing of RLHF RLHF treats a language model as a contextual bandit that receives a single scalar reward per complete response, making full Markov decision process machinery unnecessary but also hiding the dangers of reward over-optimisation.
- 09 Credit Assignment over Long Generations Explains why distributing a single scalar reward back across hundreds of generation steps is the central unsolved tension in RL for language models, and surveys the main strategies used to address it.
- 10 Evaluating RL-Tuned Models Standard NLP benchmarks break silently when applied to RL-tuned models because the training objective optimises for a reward signal that can diverge from genuine capability, requiring a distinct evaluation stack to distinguish real improvement from reward gaming.
- 11 Exploration in Language-Model RL Language-model RL training collapses silently when the policy stops generating diverse completions, and standard RL exploration techniques must be reinterpreted to work inside a token-sequence action space.
- 12 GRPO: Group Relative Policy Optimisation GRPO removes the critic network from PPO by estimating baselines from a sampled group of outputs, halving the GPU footprint while delivering competitive reasoning improvements.
- 13 Multi-Turn and Agentic RL Multi-turn and agentic RL extends single-response RLHF to sequences of actions across environment steps, requiring credit assignment, trajectory-level rewards, and new training algorithms suited to long-horizon tool-using agents.
- 14 Offline RL and the DPO Connection DPO re-derives the standard KL-regularised RLHF objective and solves it in closed form, turning preference alignment into a supervised classification loss over offline data without ever sampling from the policy during training.
- 15 PPO for Language Models Proximal Policy Optimisation clips the policy update ratio to prevent destructive gradient steps, making it the workhorse algorithm for RLHF fine-tuning of large language models.
- 16 RLVR: RL from Verifiable Rewards RLVR replaces the trained reward model in RLHF with an automated verifier that checks correctness against a ground-truth answer, producing a clean binary signal that sidesteps reward hacking and scales to tasks like maths and code where answers can be checked programmatically.
- 17 Reasoning RL and R1-Style Training How DeepSeek-R1 and Kimi k1.5 demonstrated that pure reinforcement learning on verifiable rewards can elicit chain-of-thought reasoning in LLMs without any human-labelled reasoning traces.
- 18 Reinforcement Learning from Human Feedback How preference data and PPO turn a pretrained language model into a helpful, honest, harmless assistant.
- 19 Reward Model Calibration and Drift Reward models trained on human preferences suffer from miscalibration and distribution shift, causing the optimised policy to exploit proxy scores in ways that diverge from actual human intent.
- 20 The Infrastructure of LLM RL LLM post-training via RL requires four coordinated systems running simultaneously - a policy, a reference model, a reward model, and a value function - and the design choices for each determine both what behaviours emerge and where the training breaks.
- 21 The KL-Regularised RL Objective The KL-regularised RL objective balances reward maximisation against a penalty that keeps the policy close to a reference model, preventing reward hacking while allowing genuine improvement.
Bandits & Exploration Regret, UCB, Thompson sampling, contextual bandits, and exploration under a budget. 7 concepts · 68 cards
- 01 Best-Arm Identification When the goal is to find the best option with confidence rather than to earn reward while searching, the right objective is simple regret, the problem splits into fixed-confidence and fixed-budget versions, and elimination-style algorithms need samples in proportion to the sum of inverse squared gaps.
- 02 Regret and the Exploration-Exploitation Tradeoff What regret measures, why the logarithmic lower bound says exploration can never stop entirely, and how the bandit setting differs from both supervised learning and full reinforcement learning.
- 03 Adversarial Bandits and EXP3 When rewards are chosen by an adversary rather than drawn from fixed distributions, deterministic optimism fails and the right tool is randomised exponential weighting over importance-weighted reward estimates, which guarantees regret of order the square root of T K log K against any reward sequence.
- 04 Contextual Bandits Why conditioning on features turns an intractable problem with many arms into a learning problem, how LinUCB and its relatives work, and the modelling choice that determines everything.
- 05 Deploying Bandits in Production The gap between the bandit formalism and a real system, the four assumptions that fail first, and what a deployment needs that the algorithm does not describe.
- 06 Thompson Sampling Why sampling from a posterior and acting greedily on the sample is a near-optimal exploration strategy, the property that makes it fit production systems, and where its Bayesian assumptions bite.
- 07 UCB and Optimism Under Uncertainty The optimism principle, why an upper confidence bound automatically balances exploration against exploitation, and how the bound's construction determines the algorithm's behaviour.
Offline & Model-Based RL Learning from logged data, distribution shift, conservative value estimation, world models and planning. 7 concepts · 72 cards
- 01 Decision Transformer and RL as Sequence Modelling Decision Transformer replaces value functions with a return-conditioned sequence model trained by supervised learning; it is stable and simple, and theory and benchmarks show it needs near-deterministic dynamics and cannot in general stitch suboptimal trajectories.
- 02 Conservative Value Estimation How pessimism turns an unverifiable estimate into a safe one, the mechanism behind conservative Q-learning, and why a lower bound is the right object when you cannot test.
- 03 Distribution Shift in Offline RL Why learning a policy from a fixed dataset fails in a way supervised learning does not, how value overestimation compounds through bootstrapping, and what makes this the central problem of the field.
- 04 Evaluating an Offline RL Policy Why the setting that forbids environment interaction also forbids the obvious way to compare policies, the estimators available, and why their variance is worst exactly where the decision matters.
- 05 Implicit Q-Learning and In-Sample Learning Implicit Q-learning avoids the out-of-distribution action problem by never evaluating an action outside the dataset, approximating the in-support maximum with expectile regression and extracting a policy by advantage-weighted behaviour cloning.
- 06 Model-Based RL and Compounding Error Why learning a dynamics model buys sample efficiency, how one-step errors compound over a rollout, and the design choices that keep imagination useful.
- 07 Planning with a Learned Model How to use a dynamics model at decision time rather than for training, why MPC re-plans every step, and what tree search adds when the model is exact and when it is not.
Multi-Agent RL Self-play, equilibria, credit assignment across agents, emergent coordination and non-stationarity. 6 concepts · 67 cards
- 01 Counterfactual Regret Minimisation How CFR splits the regret of a whole imperfect-information game into independent local regrets at each information set, why the average strategy converges to a Nash equilibrium only in two-player zero-sum games, and how CFR+, Linear CFR, Libratus and Pluribus turned the theory into superhuman poker.
- 02 Credit Assignment Across Agents Why a shared team reward gives every agent the same noisy signal, how counterfactual baselines and value factorisation isolate individual contributions, and the structural assumption each makes.
- 03 Emergent Coordination and Communication How agents develop coordinated behaviour without being told to, why learned communication protocols are hard to train and harder to interpret, and what counts as evidence of genuine communication.
- 04 Equilibria as Learning Targets Why Nash equilibrium is the wrong objective for most multi-agent learning, what the alternatives assume, and how the game's structure determines whether convergence is even possible.
- 05 Non-Stationarity in Multi-Agent Learning Why every guarantee from single-agent RL dissolves when other learners are present, what breaks in the Markov assumption, and the two architectural responses.
- 06 Self-Play and Population-Based Training Why playing against yourself generates an automatic curriculum, the cycling and forgetting failures that follow, and how a league of opponents fixes both.
05
Inference, Systems & Hardware
Where the model meets the silicon, the memory bus and the latency budget.
4tracks
67concepts
645cards
9.0hreading
Inference Optimisation KV cache, FlashAttention, speculative decoding, quantisation and continuous batching. 14 concepts · 170 cards
- 01 Evaluating Quantised Models A quantised model is meant to be a drop-in replacement, so matching aggregate accuracy is the wrong acceptance test; what you need to measure is how often its answers differ from the original's.
- 02 KV Cache Why decoder inference is quadratic without a KV cache and linear with one, and why managing that cache is now the dominant memory problem in LLM serving.
- 03 Quantisation - INT8, INT4, FP8 How to cut weight and activation precision below 16 bits without wrecking quality, and which scheme to pick for which deployment.
- 04 Structured Pruning and Sparsity Why removing half a model's weights is easy and making it run twice as fast is not, and how the field converged on layer, head, and channel removal followed by distillation.
- 05 Weight-Only Post-Training Quantisation Compressing weights to four bits while leaving activations in bf16 is the default way large models are made to fit, and GPTQ and AWQ get there by two genuinely different arguments about which errors matter.
- 06 vLLM and Continuous Batching Why static batching wastes most of your GPU on variable-length workloads, and how iteration-level scheduling combined with PagedAttention raises throughput by an order of magnitude.
- 07 Activation Outliers and Rotation-Based Quantisation A handful of channels in every transformer carry activations a hundred times larger than the rest, which is why activation quantisation fails where weight quantisation succeeds, and why rotating the hidden state fixes it.
- 08 Early Exit and Adaptive Depth Not every token needs all eighty layers, and the methods that act on this observation break batching and the KV cache in ways that decide whether the idea survives contact with a serving stack.
- 09 FlashAttention An IO-aware attention kernel that is both faster and lower-memory than the textbook implementation by tiling computation to keep activations in SRAM.
- 10 KV Cache Eviction and Compression Once the KV cache outgrows HBM the only remaining lever is to keep less of it, and the choice between dropping tokens, quantising them, or skipping them per query decides which capability you lose first.
- 11 Mixture-of-Experts Inference Why serving MoE models is harder than serving dense models of equivalent quality, and how DeepSeek and Mistral made it work in production.
- 12 Paged Attention and the KV Memory Manager How vLLM's PagedAttention treats the KV cache like OS virtual memory, cutting fragmentation waste from 60-80% to under 4% and roughly doubling to quadrupling serving throughput.
- 13 Speculative Decoding Use a small draft model to propose tokens that a large verifier accepts or rejects in parallel, giving lossless 2-3x latency wins on autoregressive generation.
- 14 Ternary and Extreme Low-Bit Models Below about three bits, quantising a trained model stops working and you have to train in low precision from the start, which changes the arithmetic of inference from multiply-accumulate to add-subtract.
Accelerator Architecture The memory wall, roofline analysis, GPU execution model, interconnects and systolic arrays. 20 concepts · 140 cards
- 01 Collective Communication Primitives The six core multi-GPU communication patterns (broadcast, reduce, all-reduce, all-gather, reduce-scatter, all-to-all) determine whether a distributed training job spends most of its time computing or waiting on the wire.
- 02 Compute-Bound vs Memory-Bound Kernels A kernel's performance ceiling is determined by whether FLOPs or memory bandwidth runs out first, and misidentifying this wastes orders-of-magnitude optimisation effort.
- 03 Floating-Point Formats for ML ML accelerators expose a menu of floating-point formats that trade numerical range and precision for throughput and memory bandwidth; choosing the wrong one silently degrades accuracy or leaves peak FLOPS on the table.
- 04 HBM Bandwidth and Capacity High Bandwidth Memory sets a hard ceiling on how fast a GPU can feed its compute units, and most LLM operations live squarely against that ceiling.
- 05 InfiniBand and Inter-Node Networking InfiniBand provides low-latency, high-bandwidth RDMA links between GPU nodes, and understanding its topology and collective communication patterns is essential for diagnosing and eliminating the network bottleneck in large-scale training.
- 06 KV-Cache Memory and Bandwidth The key-value cache trades GPU memory capacity for inference speed, and understanding how that trade interacts with memory bandwidth is what separates fast serving systems from slow ones.
- 07 NVLink and Intra-Node Interconnect NVLink is NVIDIA's proprietary GPU-to-GPU interconnect that delivers up to 900 GB/s aggregate bandwidth on H100, replacing PCIe as the bottleneck in multi-GPU training by making all-reduce and tensor parallelism far cheaper.
- 08 Occupancy and Latency Hiding GPU occupancy measures how many warps are resident on a streaming multiprocessor relative to its hardware maximum, and high occupancy is the primary mechanism by which the GPU hides memory and arithmetic latency to sustain throughput.
- 09 Power, Thermals, and Clock Throttling GPU accelerators operate under hard power and thermal budgets that silently reduce clock speeds mid-workload, making sustained throughput lower than peak spec sheets advertise.
- 10 Prefill vs Decode LLM inference splits into two hardware-distinct phases - a compute-bound prefill that processes all prompt tokens in parallel, and a memory-bandwidth-bound decode that generates tokens one at a time, each with fundamentally different bottlenecks on the same GPU.
- 11 Reading an Accelerator Datasheet A datasheet number means nothing without the four unit-aware ratios that reveal whether your workload will actually be compute-bound or memory-bound on that chip.
- 12 TPU Systolic Arrays A systolic array is a grid of multiply-accumulate units wired to pass partial sums directly between neighbours, letting Google's TPU sustain 92 TOPS on matrix multiplication without repeatedly hitting off-chip memory.
- 13 Tensor Cores and Matrix Engines Tensor Cores are specialised matrix-multiply-accumulate units on modern GPUs that deliver peak FLOP/s only when operand shapes and numeric formats are chosen correctly.
- 14 The FLOPs of a Transformer Forward Pass A systematic derivation of how many floating-point operations a single transformer forward pass costs, and why that number dictates hardware choice, batch strategy, and scaling decisions.
- 15 The GPU Execution Model GPUs execute thousands of threads in lockstep groups called warps; understanding that hierarchy and where threads stall is the single most important mental model for writing fast GPU code.
- 16 The GPU Memory Hierarchy A GPU's memory is a multi-tier hierarchy where bandwidth drops and latency rises by orders of magnitude as you move outward from registers to HBM, and the speed of your kernel is almost always determined by which tier bottlenecks it.
- 17 The Memory Wall and Arithmetic Intensity Arithmetic intensity determines whether a GPU kernel is memory-bound or compute-bound, and almost every LLM inference operation sits on the wrong side of that line.
- 18 The Roofline Model The Roofline Model bounds attainable hardware performance using two ceilings - peak compute throughput and peak memory bandwidth - letting you diagnose whether a kernel wastes silicon or wasits time waiting for data.
- 19 Why GEMMs Dominate Almost every compute-heavy operation in a neural network reduces to a matrix multiply, which is why hardware and compilers optimise almost exclusively for GEMM throughput.
- 20 Hardware Cost of Mixture-of-Experts Sparse MoE models reduce FLOPs per token but introduce all-to-all communication, load-imbalance penalties, and memory pressure that can erase those savings unless the system is carefully co-designed.
Kernels & Compilers CUDA, Triton, fusion, tiling, torch.compile, CUDA graphs and roofline-guided optimisation. 20 concepts · 140 cards
- 01 What a CUDA Kernel Is A CUDA kernel is a C++ function that runs simultaneously on thousands of GPU threads, each identified by a coordinate in a structured grid, and understanding this execution model is the prerequisite for reasoning about throughput in any deep-learning workload.
- 02 Graph Capture and CUDA Graphs CUDA Graphs record a sequence of GPU operations as a reusable execution graph, eliminating per-kernel CPU launch overhead and enabling significant throughput gains for workloads with static shapes and control flow.
- 03 Kernel Fusion Kernel fusion eliminates redundant memory round-trips by merging multiple GPU operations into a single kernel launch, turning memory-bandwidth bottlenecks into throughput wins.
- 04 Memory Coalescing Memory coalescing is the hardware mechanism by which a GPU groups multiple thread memory requests into a single wide transaction, and writing kernels that exploit it is often the single largest lever on throughput.
- 05 Operator Lowering and IRs Operator lowering is the process of progressively translating high-level tensor operations through a sequence of intermediate representations until hardware-executable instructions are produced.
- 06 Profiling GPU Workloads Profiling a GPU workload means measuring where time and memory bandwidth actually go, so that optimisation effort lands on the real bottleneck rather than a guess.
- 07 Shared Memory and Tiling Shared memory is a programmer-controlled on-chip SRAM that lets a thread block reuse data without re-fetching it from global memory, and tiling is the technique that makes that reuse systematic.
- 08 The CUDA Programming Model CUDA organises GPU execution into a three-level hierarchy of grids, blocks, and threads, and every performance decision traces back to how well that hierarchy is exploited.
- 09 Triton: Python-Level GPU Kernels Triton lets you write GPU kernels in Python by operating on tiles of data rather than individual threads, and its compiler handles shared-memory management, coalescing, and vectorisation automatically.
- 10 When Not to Write a Custom Kernel Writing a CUDA kernel is expensive to maintain and easy to get wrong; this concept maps the decision boundary between writing one and leaning on existing compilers and libraries.
- 11 Writing a Fused Softmax A fused softmax kernel collapses three separate memory-bound passes over a matrix row into one, cutting HBM traffic by roughly 4x and turning a memory-bound operation into a compute-limited one.
- 12 XLA and Just-In-Time Compilation XLA compiles a whole computation graph into fused, hardware-specific kernels at runtime, trading a one-time compilation cost for sustained throughput gains across GPUs and TPUs.
- 13 torch.compile and TorchInductor torch.compile traces PyTorch graphs at runtime via TorchDynamo, then lowers them through TorchInductor to fused Triton or C++ kernels, delivering 20-36% throughput gains with no model rewrites.
- 14 Autotuning GPU Kernels Autotuning systematically searches a discrete configuration space of tile sizes, warp counts, and pipeline stages to find the fastest kernel for a given GPU and problem shape, replacing manual heuristics with empirical benchmarking.
- 15 Custom Kernels for Mixture-of-Experts MoE models break the dense-GEMM assumption that GPU libraries are optimised for, so efficient inference requires custom grouped-GEMM and block-sparse kernels that handle variable-length expert batches without padding or token dropping.
- 16 Mixed-Precision Kernels Mixed-precision kernels reduce memory bandwidth and arithmetic cost by storing and computing in lower-precision formats while selectively preserving full precision where numerical stability demands it.
- 17 Paged Attention as a Memory Manager PagedAttention borrows the OS virtual-memory paging model to eliminate KV-cache fragmentation, letting a single GPU serve far more concurrent requests than contiguous allocation allows.
- 18 Quantised GEMM Kernels Quantised GEMM kernels replace 16-bit or 32-bit matrix multiplications with 8-bit or 4-bit integer arithmetic, cutting memory bandwidth and compute cost while preserving model accuracy through careful scaling and outlier handling.
- 19 Roofline-Guided Kernel Optimisation The roofline model maps a kernel's arithmetic intensity against hardware ceilings to diagnose whether compute or memory bandwidth is the binding constraint, and directs every subsequent optimisation decision.
- 20 Why FlashAttention Is a Kernel Story FlashAttention achieves its speedups not by reducing FLOPs but by restructuring the attention computation into a single tiled CUDA kernel that fits working data in on-chip SRAM, eliminating the dominant cost of round-tripping through GPU HBM.
Serving Systems Prompt caching, gateways and routing, token accounting, and multi-tenant isolation. 13 concepts · 195 cards
- 01 Autoscaling and Cold Starts in LLM Serving Why GPU utilisation is a useless autoscaling signal for LLM servers, what a cold start actually costs, and how to scale a fleet whose new replicas take minutes to become useful.
- 02 LLM Gateways and Routing Why every serious LLM deployment ends up behind a gateway, and how to choose between LiteLLM, Portkey, OpenRouter, and rolling your own.
- 03 Model Rollouts: Shadow, Canary, Rollback Why swapping the model behind a product is nothing like deploying code, how shadow traffic and canary stages catch what offline evals miss, and what has to be versioned together for rollback to mean anything.
- 04 Prompt Caching Infrastructure How Anthropic, OpenAI, and vLLM let you reuse the KV cache of repeated prefixes, what the cache key actually is, and the patterns that turn cache hit rate into a real bill reduction.
- 05 Semantic Caching and Response Reuse Returning a stored answer when a new prompt is merely similar to an old one converts an inference call into a vector lookup, and turns a correctness question into a similarity-threshold setting.
- 06 Serving SLOs: TTFT, TPOT and Goodput Why tokens per second is the wrong number to optimise, how TTFT and TPOT split the latency budget, and what goodput measures that throughput hides.
- 07 Token Accounting, Billing, and Quotas Why a single token counter is not enough, how to attribute spend across users and features without losing your mind, and the patterns that prevent one bad actor from spending the whole month's budget on a Tuesday afternoon.
- 08 Admission Control and Load Shedding Why an overloaded LLM server degrades for everyone at once, how token-aware admission control keeps latency SLOs intact by rejecting work early, and what graceful degradation looks like when you would rather not say no.
- 09 Disaggregated Prefill and Decode Serving Why prefill and decode want opposite hardware and parallelism, how splitting them across separate GPU pools raises goodput, and what the KV cache transfer costs.
- 10 Multi-Tenant Serving and Isolation Serving many tenants from one model is cheap and easy; giving each tenant their own fine-tune is expensive and hard. S-LoRA and per-request LoRA serving collapse the trade-off, but only for tenants who can share a base model.
- 11 Prefix-Aware Routing and KV Cache Reuse Why load-balancing LLM requests round-robin throws away computed KV cache, and how routing on prompt prefix turns a fleet's caches into a shared asset.
- 12 Queueing Theory for LLM Serving Little's Law fixes the maximum request rate a batched LLM server can sustain at a given latency, and the heavy-tailed distribution of output lengths explains why queues form long before the GPU is saturated.
- 13 Request Scheduling and Preemption Shortest-job-first minimises average waiting time and requires knowing job length, which an LLM server cannot know; the workarounds are predicting the rank of output lengths or preempting at token granularity.
06
Applied LLM Engineering
Building things people use: retrieval, prompts, agents and production architecture.
6tracks
63concepts
652cards
8.2hreading
Retrieval & RAG Vector stores, hybrid retrieval and reranking, and when to retrieve instead of fine-tune. 14 concepts · 161 cards
- 01 Choosing and Adapting an Embedding Model Why the top of the MTEB leaderboard is a bad way to pick an embedding model, what dimension, context length, and asymmetry actually cost you in production, and when fine-tuning on your own hard negatives beats buying a bigger model.
- 02 Chunking Strategies for Retrieval Why the unit you index decides the ceiling on retrieval quality, how fixed, recursive, semantic and contextual chunking differ, and what each one loses.
- 03 Contextual Retrieval and Chunk Augmentation Chunks lose the context that made them meaningful; prepending a short LLM-written situating sentence to each chunk before embedding cuts retrieval failures by roughly a third, and the technique only became affordable because of prompt caching.
- 04 Fine-tuning vs RAG When to teach the model new behaviour vs when to retrieve fresh context at runtime.
- 05 Hybrid Retrieval - BM25 + Vector + Reranking Why pure vector search misses exact-match queries, how RRF combines lexical and semantic results, and where a cross-encoder reranker buys back the precision you lost.
- 06 Query Transformation for Retrieval Why the user's question is often a bad search query, and how rewriting, decomposition, multi-query fan-out and HyDE close the gap between how people ask and how documents are written.
- 07 RAG Evaluation and Groundedness How to separate retrieval failures from generation failures, which metrics actually diagnose each stage, and why groundedness is measurable while helpfulness mostly is not.
- 08 Reranking and Cross-Encoders Why a second-stage model that reads the query and document together fixes most retrieval failures, what it costs in latency, and how to size the candidate set.
- 09 Retrieval Augmented Generation The end-to-end RAG pipeline from chunking through retrieval, reranking, and grounded generation.
- 10 Vector Databases Compared - pgvector, Qdrant, Milvus, Weaviate, LanceDB A practitioner's guide to picking a vector store, weighing index trade-offs against the operational cost of running yet another database alongside your primary store.
- 11 ANN Indexes: HNSW, IVF and PQ How approximate nearest neighbour indexes trade recall for latency and memory, what HNSW, IVF-PQ, ScaNN and DiskANN each optimise for, and why recall is a knob rather than a property.
- 12 Agentic and Iterative Retrieval Why a single retrieve-then-read pass cannot answer questions whose second search depends on the first result, and how retrieve-reason interleaving, self-reflection tokens, and search agents trade latency and cost for multi-hop accuracy.
- 13 GraphRAG and Community Summarisation Why top-k retrieval cannot answer "what are the main themes in this corpus", how building an entity graph and pre-summarising its communities turns a global question into a map-reduce over summaries, and what that indexing bill buys you.
- 14 Late Interaction and Multi-Vector Retrieval How ColBERT-style models keep one vector per token instead of one per document, why MaxSim recovers most cross-encoder quality at index-time cost, and what the storage bill looks like.
Prompt Engineering In-context learning, chain of thought, structured output, compression and injection-aware design. 10 concepts · 94 cards
- 01 Chain of Thought Prompting Why telling the model to think step by step radically improves reasoning, and when it actively hurts.
- 02 Few-Shot and In-Context Learning Learning a task from a handful of worked examples placed in the prompt, with no weight updates, and the surprising evidence about what those examples actually teach.
- 03 Injection-Aware Prompt Design How to structure prompts that consume untrusted input so injection is harder, and why prompt design alone can never make an LLM injection-proof.
- 04 Multimodal Prompting What changes when part of the prompt is an image: token cost scales with resolution, ordering of image and text changes the answer, and the reliable failure mode is confident description of objects that are not there.
- 05 Prompt Chaining and Task Decomposition Splitting a hard task into a pipeline of simpler, individually-checkable prompts so each step can be validated, routed, and debugged on its own.
- 06 Structured Output Coercion How to coax reliable JSON, XML, and tabular output from a model using prompting alone, and why that gives you no hard guarantee the way constrained decoding does.
- 07 System Prompt Design What the system prompt actually buys you as a privileged, cache-stable, always-present segment, how to structure it, and why length in that slot costs more than it looks.
- 08 Automatic Prompt Optimisation Treating the prompt as a parameter to be searched rather than a string to be tweaked, using a metric, a dataset, and an LLM that proposes and critiques its own instructions.
- 09 Prompt Compression Cutting prompt tokens while holding task performance, via perplexity-based token dropping (LLMLingua) or learned gist tokens, and when prompt caching beats both.
- 10 Prompt Format Sensitivity Semantically identical prompts that differ only in separators, spacing, or option order can move accuracy by tens of points, which makes any single-format benchmark number a sample rather than a measurement.
Agents & Tool Use Function calling, ReAct loops, MCP, agent memory architectures and evaluation harnesses. 15 concepts · 169 cards
- 01 Agent Frameworks Compared LangGraph, CrewAI, AutoGen, and the OpenAI Agents SDK solve different problems; the harder question is whether you need a framework at all.
- 02 Agent Skills and Progressive Disclosure Packaging agent expertise as folders of instructions and scripts that load in layers, so a hundred specialisations cost a few hundred tokens until one of them is actually needed.
- 03 Agent-to-Agent Interoperability MCP standardises how one agent reaches tools and data; A2A standardises how two independently built agents discover each other and collaborate as peers, which is a different problem with a different failure surface.
- 04 Model Context Protocol (MCP) The open standard that replaces bespoke per-tool integrations with one protocol, so any compliant client can talk to any compliant server.
- 05 Planning and Task Decomposition in Agents Why an LLM that reasons well step by step still fails to produce a valid multi-step plan, and how decomposition, external planners, and replanning close the gap.
- 06 Tool Use and Function Calling How models invoke external tools to fetch data, run code, and take actions in the world.
- 07 Agent Evaluation Harnesses Single-output accuracy says nothing about an agent that takes thirty steps; evaluating agents means scoring trajectories, environment state, and reliability across runs.
- 08 Agent Memory Architectures An agent whose only memory is its context window is amnesiac between sessions; persistent memory is the architecture that decides what to keep, where, and how to retrieve it.
- 09 Agentic AI and ReAct From single tool calls to multi-step agents that plan, act, observe, and recover from errors.
- 10 Code Execution as a Tool Interface Instead of calling tools one at a time through the context window, the agent writes code against tool APIs in a sandbox, which cuts both tool-definition overhead and intermediate results out of the token budget.
- 11 Computer-Use Agents: Operating a GUI Through Pixels How agents that click and type on a real desktop differ from tool-calling agents, why GUI grounding is the bottleneck, and what OSWorld measured that API benchmarks cannot.
- 12 Durable Agent Execution and Recovery Long-running agents fail mid-task for mundane reasons, so the loop needs checkpointed state, idempotent side effects, and the ability to resume from a step rather than restart from the prompt.
- 13 Long-Horizon Agent Reliability Why per-step accuracy compounds into task failure, how METR's time-horizon metric reframes agent capability, and which architectural moves actually raise the exponent.
- 14 Orchestrator-Worker Subagent Architectures A lead agent decomposes a task and spawns subagents with clean context windows that explore in parallel and return compressed summaries, buying breadth and context isolation at a large token cost and a coordination risk.
- 15 Sandboxing and Least Privilege for Agents Why agent security has to be enforced outside the model, how capability scoping and human-in-the-loop gates work, and what the CaMeL design proves about the limits of prompting.
Claude Certified Architect Agentic loops, coordinator-subagent designs, tool interfaces and reliability patterns for Claude. 10 concepts · 82 cards
- 01 Agentic Loops and stop_reason Handling The agentic loop lifecycle - sending requests, inspecting stop_reason, executing tools, and appending results. The foundation of every autonomous Claude agent.
- 02 CLAUDE.md Configuration and Claude Code Workflows CLAUDE.md hierarchy, .claude/rules/ with glob patterns, custom commands and skills, plan mode vs direct execution, and CI/CD integration.
- 03 Cost and Throughput Engineering on the Claude API The levers that actually move an LLM bill — effort, batch processing, model tiering, task budgets, and deferred tool loading — what each one costs in latency or quality, and why measuring before tuning is not optional.
- 04 Programmatic Enforcement vs Prompt-Based Guidance When to use hooks and programmatic prerequisites for guaranteed compliance versus system prompt instructions for probabilistic guidance.
- 05 Prompt Engineering and Structured Output Patterns Explicit criteria, few-shot prompting, tool_use with JSON schemas, validation-retry loops, and the Message Batches API.
- 06 Tool Interface Design and MCP Integration Writing effective tool descriptions, structured error responses, MCP server scoping, and the distinction between MCP tools and resources.
- 07 Context Lifecycle: Editing, Compaction, and Memory Three different mechanisms manage a long-running Claude agent's context — clearing stale tool results, summarising history server-side, and persisting files across sessions — and choosing wrongly between them is one of the most common architecture mistakes.
- 08 Context Management and Reliability Patterns Context preservation across long interactions, escalation decision-making, error propagation in multi-agent systems, and information provenance.
- 09 Multi-Agent Coordinator-Subagent Architecture Hub-and-spoke multi-agent design with coordinator delegation, isolated subagent context, parallel execution, and iterative refinement loops.
- 10 Prompt Caching Economics for Claude Agents Caching is a prefix match, so one interpolated timestamp can make an entire agent uncacheable; the architecture decisions that matter are ordering and stability, not where you place the breakpoint markers.
LLM Application Architecture Routing, fallbacks, caching layers, structured state, and the reference shapes production LLM apps take. 7 concepts · 74 cards
- 01 Batch and Asynchronous LLM Workloads How provider batch APIs trade a day of latency for half the price, how to shape traffic under token-bucket rate limits when you cannot wait, and why every asynchronous LLM job needs idempotent writes and a reconciliation step.
- 02 Document Ingestion and Parsing Pipelines Why turning PDFs into text is the silent ceiling on retrieval quality, how layout-model pipelines and vision-language parsers differ in cost and failure, and how to re-ingest a changing corpus without reprocessing all of it.
- 03 Fallbacks, Timeouts and Degradation How to build an LLM feature that survives a provider outage, why the usual retry patterns need adjusting for generation, and what degrading gracefully looks like when the core capability is unavailable.
- 04 Model Routing and Cascades Why sending every request to the strongest model is usually wrong, the two routing patterns and what each requires, and the economics that decide whether routing pays.
- 05 Caching Layers for LLM Applications The four distinct caches in a mature LLM system, what each requires to be correct, and why semantic caching is the one that is dangerous.
- 06 Structured Output and Constrained Generation Why parsing free text is the wrong integration point, how constrained decoding guarantees valid syntax, and what a guarantee about form does not give you about content.
- 07 Where State Lives in an LLM Application Why the context window is the worst place to keep state, the four stores a mature application actually uses, and the assembly step that decides what the model sees.
AI for Software Engineering Code models, repository context, patch generation, test-driven agents and SWE benchmarks. 7 concepts · 72 cards
- 01 Automated Program Repair and Fault Localisation How spectrum-based fault localisation ranks suspicious code from test coverage, why generate-and-validate repair overfits weak test suites, and how LLM pipelines such as Agentless rebuilt the same localise-repair-validate loop.
- 02 Code Generation Benchmarks and pass@k How HumanEval turned code evaluation into execution, why the unbiased pass@k estimator exists and how it is derived, and why saturation and contamination pushed the field toward time-stamped benchmarks.
- 03 Code Review by Model Why review is a better fit for current models than authoring, the precision problem that determines whether it is used, and the classes of finding worth reporting.
- 04 Fill-in-the-Middle and Completion Models Why a left-to-right model cannot do the most common editing task, the training transformation that fixes it, and the latency budget that shapes everything about inline completion.
- 05 Repository Context and the Retrieval Problem Why a model with a large context window still cannot see a codebase, what makes code retrieval different from document retrieval, and the signals that actually locate relevant code.
- 06 SWE-bench and Agentic Coding Evaluation What resolving a real GitHub issue measures that a function-completion benchmark does not, the contamination and scoping problems that complicate it, and how to read a reported score.
- 07 Test-Driven Agents and Verification Loops Why an executable check transforms an agent's reliability, how to structure the loop so it converges, and the failure where the agent optimises the test rather than the code.
07
Reasoning, Evaluation & Safety
Models that think longer, the evals that measure them, and the failure modes that matter.
3tracks
54concepts
748cards
7.3hreading
Reasoning Models Test-time compute, process reward models, the o-series, DeepSeek-R1 and contamination. 15 concepts · 196 cards
- 01 Chain of Thought, Self-consistency, and Search at Inference A tour of the inference-time reasoning toolkit - from zero-shot CoT prompts to MCTS-decoded reasoning trees, and when each pays for itself.
- 02 Hybrid Reasoning Models One set of weights that can answer instantly or think at length, how mode switching and thinking budgets are trained in, and why the industry converged on hybrids over separate reasoning models.
- 03 OpenAI o1, o3 and the Reasoning-Model Family What is publicly known and what is speculated about OpenAI's reasoning line, why the chain of thought is hidden, and what o3's ARC-AGI result actually proved.
- 04 Reasoning Evals and the Contamination Problem A guided tour of the reasoning benchmark canon, why each saturated faster than the field expected, and the move to live held-out evals as the contamination crisis bites.
- 05 Test-time Compute Scaling Why "thinking longer" at inference can substitute for "training bigger", how the trade-off is operationalised as a token budget, and where the strategy stops paying.
- 06 Chain-of-Thought Faithfulness Whether a model's stated reasoning is the reasoning that produced its answer, how the hint-injection test measures it, and why unfaithful CoT undermines monitoring more than it undermines accuracy.
- 07 Controlling Reasoning Length The mechanisms that set how long a model thinks, from s1's crude budget forcing to length-targeted RL, and why overthinking makes length control a quality problem rather than just a cost problem.
- 08 DeepSeek-R1 and the Open Reasoning Recipe How DeepSeek's R1 pipeline produced o1-class reasoning with an open paper, an open model, and a recipe other labs could replicate within weeks.
- 09 Deliberative Alignment Teaching a model the text of its own safety policy and training it to reason over that text before answering, which improves jailbreak robustness and overrefusal at the same time.
- 10 Inference-Time Scaling Laws and Thinking Budgets How accuracy trades against tokens spent at inference, why the optimal strategy depends on question difficulty, and what a thinking budget actually buys.
- 11 Process Reward Models and Verifiable Rewards Why scoring every step of a reasoning trace beats scoring only the final answer, and how Ai2 and DeepSeek replaced PRMs entirely with programmatic correctness checks.
- 12 Reasoning Distillation and Budget Forcing A thousand carefully chosen reasoning traces and a decoding trick can move a 32B model past o1-preview on competition maths, which says something uncomfortable about what reasoning training is actually teaching.
- 13 The Limits of Self-Correction Why asking a model to review its own answer often makes it worse, what separates intrinsic from extrinsic correction, and the oracle-label leak that inflated the early results.
- 14 Tool-Integrated Reasoning Interleaving code execution with natural-language reasoning removes an entire class of error the model cannot fix by thinking harder, and creates a new class of error it cannot see.
- 15 Verifier-Guided Search and Best-of-N Sampling many candidate solutions is only useful if you can pick the right one, and the gap between what sampling can reach and what selection can find is the central constraint on test-time scaling.
Evaluation & MLOps Benchmarks, LLM-as-judge, red-teaming, model registries, drift detection and observability. 14 concepts · 144 cards
- 01 Arena Elo and Preference Ranking How pairwise human votes become a leaderboard through the Bradley-Terry model, what the Elo framing gets wrong, and the selection effects that distort arena rankings.
- 02 Custom Evals and LLM-as-Judge Why every production team eventually builds its own eval set, and how to use LLM judges without being fooled by their well-documented biases.
- 03 Epistemic and Aleatoric Uncertainty in LLMs Separating "the model does not know" from "the question has several right answers", why token-level entropy conflates the two, and how semantic clustering fixes it.
- 04 Eval-Driven Development How teams turn production failures into a regression suite, why the first fifty examples matter more than the framework, and the discipline that keeps an eval set honest as the product changes.
- 05 HELM and Holistic Evaluation Why a single accuracy number is gameable, and how Stanford's HELM, BIG-bench, and lm-evaluation-harness push evaluation toward a multi-axis picture.
- 06 LLM Observability Tooling How tracing an LLM app captures the full fan-out of model and tool calls behind one user request, and how LangSmith, Langfuse, Helicone, and Phoenix differ in what they instrument.
- 07 Model Registry, Lineage, and Reproducibility The infrastructure that answers "which dataset, code, and hyperparameters produced this checkpoint?" - and why you only miss it the first time you cannot reproduce a model.
- 08 Production Monitoring and Drift Detection How to catch silent regressions in deployed LLMs by monitoring input drift, output quality, and per-user randomised experiments before users tell you something is broken.
- 09 Public Benchmarks - MMLU, GPQA, HumanEval, MATH A tour of the academic benchmarks that anchor frontier model launches, and why most of them are saturating, contaminated, or both.
- 10 Selective Prediction and Abstention Letting a system answer only when it is likely to be right, measuring the result with a risk-coverage curve rather than accuracy, and choosing the threshold from the cost of being wrong.
- 11 Agentic Benchmarks: SWE-bench and Its Descendants What changes when a benchmark task requires many steps in a real environment, why execution-based grading is the whole point, and the contamination and harness confounds that make agent scores hard to compare.
- 12 Conformal Prediction for LLMs A distribution-free procedure that converts any confidence score into prediction sets with a finite-sample coverage guarantee, and what that guarantee does and does not promise once the predictor is a language model.
- 13 Error Bars for Evals: Sampling, Clustering and Paired Tests Why a benchmark score is an estimate with a standard error, how clustering inflates it, and why paired comparison is the single highest-leverage change to an eval report.
- 14 Red-Teaming and Adversarial Evaluation Why benign benchmark scores do not predict how a deployed model behaves under attack, and the human and automated methods used to find the failures first.
Safety & Alignment Prompt injection, jailbreaks, Constitutional AI, reward hacking and mechanistic interpretability. 25 concepts · 408 cards
- 01 Activation Steering and Representation Engineering Adding a direction to a model's residual stream at inference time changes its behaviour without any weight update, which makes concepts like refusal, sycophancy and sentiment into vectors you can add, scale, or subtract.
- 02 Alignment Evaluations and Frontier-Model Risk How frontier labs and governments measure dangerous capabilities, what an eval-gated release looks like, and where the regulatory regime sits in 2026.
- 03 Guardrail Classifiers and Content Filtering Putting a separate classifier in front of and behind the model gives you a safety layer updatable in hours rather than retrained over weeks, and it buys that agility with latency, false refusals, and a second model to keep honest.
- 04 Jailbreaks and Refusal Robustness How attackers reliably bypass model refusal training, why post-hoc filters are necessary but never sufficient, and how AILuminate measures what remains.
- 05 Prompt Injection Why LLMs cannot reliably tell instructions from data, how indirect injection weaponises retrieved content, and which partial defences are worth deploying.
- 06 Training Data Memorisation and Extraction Language models reproduce fragments of their training corpus verbatim, and an adversary with only API access can pull that data back out, which turns a training-set decision into a permanent disclosure risk.
- 07 Watermarking and Content Provenance How a statistical signal is embedded in generated text without changing its quality, why SynthID-Text scaled to production, and the robustness limits every scheme shares.
- 08 AI Control: Safety Without Trusting the Model The research agenda that assumes the model may be deliberately subverting your safeguards, and designs protocols with a red team that gets to try.
- 09 Activation Patching and Causal Tracing Swap one internal activation from a corrupted run into a clean run and measure how much of the output moves; done systematically it localises where in a network a specific computation happens.
- 10 Alignment Faking A model that infers it is in training, and that complying now prevents its values from being modified later, has an instrumental reason to comply that has nothing to do with actually holding those values.
- 11 Circuit Tracing and Attribution Graphs Replacing a model's MLP layers with a cross-layer transcoder produces a differentiable stand-in whose feature-to-feature influences can be read off as a graph, turning "which features are active" into "which features caused which".
- 12 Constitutional AI and RLAIF How Anthropic replaced human harmlessness labels with a written constitution and a critique-and-revise loop, and why this makes alignment auditable.
- 13 Data Poisoning and Backdoor Attacks An adversary who controls a small slice of the training corpus can install a trigger-activated behaviour that survives ordinary evaluation, and the number of documents required turns out not to grow with model scale.
- 14 Debate as Scalable Oversight If two strong models argue for opposing answers in front of a weaker judge, the honest side should have the easier case, which would let humans supervise systems whose answers they cannot verify directly.
- 15 Differential Privacy for Language Models The only training-time privacy defence with a formal guarantee, achieved by clipping per-example gradients and adding calibrated noise, and the concrete capability price that guarantee costs.
- 16 Evaluation Awareness and Sandbagging A safety evaluation assumes the subject cannot tell it is being evaluated, and frontier models increasingly can; that breaks the inference from a good test score to safe deployment behaviour.
- 17 Machine Unlearning in Language Models What it means to remove knowledge from a trained model, why WMDP and TOFU measure different things, and the relearning attacks that show most unlearning is suppression.
- 18 Mechanistic Interpretability Primer How sparse autoencoders extract human-interpretable features from model activations, what circuit-level analysis buys you for safety, and where the science is still contested.
- 19 Membership Inference on Language Models Deciding whether a specific document was in a model's training set is the canonical privacy attack, and on large language models it works far worse than the classical literature predicts, for reasons that are themselves informative.
- 20 Model Extraction and Weight Stealing A black-box API leaks more about the model behind it than intended, and with the right queries an attacker can recover architectural secrets or a functional clone for a tiny fraction of the training cost.
- 21 Model Organisms of Misalignment and Sleeper Agents Why safety researchers deliberately build misaligned models, what the sleeper-agent experiments showed about the durability of backdoors, and why adversarial training made things worse.
- 22 SAE Limitations and Crosscoders Sparse autoencoders were adopted on the premise that they recover a model's atomic, complete feature vocabulary, and three results published in 2024 and 2025 show they recover neither.
- 23 Scalable Oversight and Weak-to-Strong Generalisation How you supervise a model on tasks you cannot evaluate yourself, why weak labels still elicit strong capabilities, and where the analogy to superhuman supervision leaks.
- 24 Sparse Autoencoders for Feature Extraction Dictionary learning applied to activations, pulling a small set of interpretable, steerable features out of neurons that individually mean nothing.
- 25 Sycophancy, Deception, and Reward Hacking Why preference-trained models learn to please rather than to be right, what alignment faking is, and why evaluating during training can mislead you.
08
Multimodal & Applications
Beyond text — vision, speech, robotics and scientific discovery.
5tracks
74concepts
669cards
9.8hreading
Vision & Multimodal ViT, CLIP, diffusion, SAM, and the vision-language models that read images as tokens. 12 concepts · 131 cards
- 01 Contrastive Vision-Language: CLIP How a 400M image-text contrastive objective produced a shared embedding space that does zero-shot classification, retrieval, and grounding without any task-specific labels.
- 02 Multimodal Hallucination and Benchmark Validity Why vision-language models describe objects that are not in the image, and why a large share of multimodal benchmark scores can be reproduced without showing the model any image at all.
- 03 Native-Resolution Vision Encoding Why squashing every image to a fixed square destroyed text legibility in early VLMs, and how tiling, patch packing and dynamic resolution replaced it at the cost of an unbounded visual token budget.
- 04 Segment Anything (SAM) and Dense Prediction How promptable segmentation became a foundation-model task, what SAM's encoder-decoder split was designed for, and where it still loses to specialist models.
- 05 Video, Audio, and Any-to-Any Models How Whisper, V-JEPA, Sora-class video generators, MusicGen, and unified any-to-any models extend the multimodal stack beyond static images.
- 06 Vision Transformers (ViT) How treating an image as a sequence of patches let pure transformers beat CNNs once data crossed the 300M-image mark, and what the architecture gave up to get there.
- 07 Visual Document Retrieval Why the OCR-parse-chunk-embed pipeline loses exactly the documents that matter, and how late-interaction retrieval over page images replaces five brittle stages with one model.
- 08 Diffusion Models How learning to invert a noise process became the dominant generative recipe for images, video, and audio, and why Flow Matching and DiTs are reshaping the recipe in 2024.
- 09 Discrete Visual Tokenisers How VQ-VAE, VQGAN and FSQ turn an image into a short sequence of integers, why codebook collapse wrecks half of them, and what the compression ratio costs in reconstruction fidelity.
- 10 Early-Fusion Mixed-Modal Models The difference between bolting a vision encoder onto a frozen LLM and training one transformer over interleaved image and text tokens from scratch, and why the second approach destabilises training.
- 11 Flow Matching and Rectified Flow Why regressing a velocity field along straight noise-to-data paths replaced score matching in frontier image models, and what the straight-line claim does and does not guarantee.
- 12 Multimodal LLMs: LLaVA, Flamingo, GPT-4V The vision-encoder-plus-projector-plus-LLM recipe that dominates open multimodal models, why Flamingo's perceiver design still matters for video, and what native-multimodal frontier models do differently.
Speech Recognition Spectrograms, CTC, RNN-T, Conformer, Whisper, streaming, diarisation and self-supervised audio. 20 concepts · 140 cards
- 01 Audio Features and Spectrograms Raw audio waveforms are rarely fed directly to speech models; this concept explains how and why they are first converted into spectrogram-based representations that compress perceptual information into a learnable 2-D grid.
- 02 The ASR Problem and Pipeline Automatic speech recognition converts a raw audio waveform into a word sequence by solving an alignment problem that classical NLP never had to face.
- 03 Beam Search Decoding in ASR Beam search decoding navigates the exponentially large label sequence space in ASR by maintaining a fixed-width frontier of the most probable partial hypotheses at each step, making it the default inference strategy for CTC, RNN-T, and attention-based models.
- 04 CTC: Connectionist Temporal Classification CTC is a training objective that lets a neural network learn to align variable-length audio to text without any hand-labelled frame-level annotations.
- 05 Causal and Chunked Attention for Streaming Streaming ASR requires attention mechanisms that never look at future audio; causal masking and chunked attention are the two principal techniques, each trading latency against accuracy in different ways.
- 06 Endpointing and Voice Activity Detection Endpointing and voice activity detection are the mechanisms that decide when a user has finished speaking, directly controlling the latency and correctness of every streaming ASR system.
- 07 Listen, Attend and Spell LAS is a purely sequence-to-sequence ASR model that replaces HMMs, CTC, and explicit pronunciation dictionaries with a pyramidal RNN encoder and an attention-based character decoder trained end-to-end.
- 08 Robustness to Noise and Accents How modern ASR systems are trained and adapted to handle environmental noise, channel distortions, and speaker accent variability without collapsing to near-zero accuracy.
- 09 Speaker Diarisation Speaker diarisation segments an audio recording into speaker-homogeneous regions and assigns each region a speaker identity, answering the question "who spoke when" without necessarily transcribing what was said.
- 10 Streaming vs Offline ASR Streaming ASR emits partial transcripts incrementally as audio arrives, trading access to future context for low latency, while offline ASR processes the full utterance and consistently achieves lower word error rates.
- 11 The CTC Blank Token and Alignment The CTC blank token is a special output symbol that lets a neural network emit one label per time-step without needing a hand-crafted alignment between audio frames and characters.
- 12 The Conformer Architecture The Conformer interleaves convolution and multi-head self-attention inside each encoder block to capture both fine-grained local acoustic patterns and long-range sequence dependencies, achieving state-of-the-art ASR accuracy on LibriSpeech.
- 13 Whisper's Multitask Decoder Whisper conditions a single sequence-to-sequence decoder on a prefix of special tokens that specify language, task, and timestamp behaviour, allowing one model to handle transcription, translation, and language identification without any task-specific heads.
- 14 Whisper: Weakly-Supervised ASR Whisper trains a sequence-to-sequence Transformer on 680,000 hours of weakly-supervised internet audio to achieve robust multilingual speech recognition without task-specific fine-tuning.
- 15 Why Sequence Length Is Hard in Audio Audio produces roughly 100 frames per second of speech, making sequence lengths 10-50x longer than equivalent text, and this mismatch drives almost every architectural and training decision in modern ASR.
- 16 Word Error Rate and ASR Evaluation Word Error Rate measures ASR accuracy as the minimum edit distance between a hypothesis and a reference transcript, normalised by reference length, but its apparent simplicity hides a nest of normalisation choices that make numbers across papers routinely incomparable.
- 17 wav2vec 2.0 and Self-Supervised Audio wav2vec 2.0 learns speech representations from raw audio without transcripts by masking latent features and solving a contrastive task over learned discrete units, then fine-tunes on as little as ten minutes of labelled speech to reach competitive word error rates.
- 18 HuBERT and Discrete Audio Units HuBERT pre-trains a speech encoder by predicting offline k-means cluster labels for masked audio frames, producing discrete unit sequences that rival phoneme transcriptions without any text supervision.
- 19 Language-Model Fusion Language-model fusion techniques inject text-only knowledge into end-to-end ASR models at inference time or training time, and the correct method depends on how much implicit language bias the acoustic model has already absorbed.
- 20 The RNN-Transducer The RNN-Transducer is a fully neural, streaming-capable sequence transduction model that replaces CTC's conditional independence assumption with a learned label-context network, enabling accurate on-device speech recognition.
Speech Synthesis Acoustic models and vocoders, Tacotron, FastSpeech, HiFi-GAN, neural codecs and voice cloning. 20 concepts · 139 cards
- 01 The TTS Problem and Pipeline Text-to-speech converts a string of characters into a waveform through a chain of normalisation, acoustic modelling, and synthesis stages, each introducing its own failure modes.
- 02 Attention Failures in TTS Attention-based TTS systems fail in predictable ways - word skipping, repetition, and unstable alignment - and understanding the mechanics behind each failure mode is essential for building reliable speech synthesis pipelines.
- 03 Bark and Fully Generative Audio Bark is a transformer-based model that generates speech, music, and nonverbal audio from text by autoregressively predicting discrete audio codec tokens, without any phoneme pipeline or continuous acoustic model.
- 04 Duration Modelling and Alignment Duration modelling assigns how many audio frames each phoneme occupies; alignment is the mechanism that learns or infers that mapping from text-audio pairs without manual annotation.
- 05 Evaluating TTS with MOS Mean Opinion Score is the field's primary yardstick for TTS naturalness, but its reliability hinges on listener pool design, anchoring, and context choices that most papers under-report.
- 06 FastSpeech and Non-Autoregressive TTS FastSpeech eliminates the sequential mel-frame dependency of autoregressive TTS by predicting all frames in parallel, trading model simplicity for a duration predictor and a length regulator.
- 07 Griffin-Lim and Classical Vocoders Griffin-Lim is an iterative phase-recovery algorithm that converts a magnitude spectrogram back into a waveform without any learned parameters, and understanding it clarifies exactly what neural vocoders had to replace and why.
- 08 HiFi-GAN and Neural Vocoders Neural vocoders convert acoustic feature representations into raw audio waveforms, and HiFi-GAN achieves near-human quality at 167x real-time speed using a multi-scale, multi-period GAN architecture.
- 09 Neural Audio Codecs Neural audio codecs compress waveforms into discrete token sequences using learned vector quantisation, enabling language models to generate speech token-by-token.
- 10 Prosody and Style Control How TTS systems encode and manipulate pitch, duration, energy, and speaking style so that synthesised speech sounds intended rather than merely intelligible.
- 11 Residual Vector Quantisation Residual vector quantisation stacks multiple codebooks to approximate a continuous audio embedding with increasingly fine-grained corrections, making it the compression backbone of modern neural audio codecs.
- 12 Streaming TTS and Latency Streaming TTS pipelines generate and deliver audio incrementally to cut time-to-first-audio from several seconds to under 300 ms, but doing so imposes hard trade-offs on chunk size, model architecture, and prosodic coherence.
- 13 Tacotron 2 Tacotron 2 is a two-stage neural TTS pipeline that converts text to mel spectrograms with a sequence-to-sequence model, then synthesises raw audio with a conditioned WaveNet vocoder, achieving near-human MOS scores.
- 14 Text Normalisation and Phonemisation Text normalisation converts raw written text into a speakable form, and phonemisation maps those words to phoneme sequences; together they determine what a TTS system says before any audio is generated.
- 15 The Acoustic Model and Vocoder Split Modern neural TTS splits the problem into two specialised sub-networks - an acoustic model that maps text to a compact spectral representation, and a vocoder that reconstructs a full audio waveform from that representation.
- 16 The Mel-Spectrogram Interface The mel spectrogram is the agreed-upon intermediate representation that decouples acoustic modelling from waveform generation in modern TTS pipelines.
- 17 WaveNet WaveNet is a fully autoregressive convolutional neural network that models raw audio waveforms one sample at a time, achieving near-human speech quality at the cost of extremely slow sequential generation.
- 18 Diffusion Models for Speech Diffusion models iteratively denoise random Gaussian noise into speech waveforms or mel-spectrograms, achieving sample quality that matches autoregressive vocoders at a fraction of the sequential compute cost.
- 19 VALL-E: TTS as Token Language Modelling VALL-E reformulates text-to-speech as a conditional language modelling problem over discrete audio codec tokens, enabling zero-shot voice cloning from a three-second recording by treating acoustic context the same way GPT treats a few-shot text prompt.
- 20 Zero-Shot Voice Cloning Zero-shot voice cloning synthesises speech in the voice of an unseen speaker from a short reference recording, without any fine-tuning at inference time.
Robotics & Embodied AI Vision-language-action models, action tokenisation, diffusion policies and sim-to-real. 10 concepts · 122 cards
- 01 Action Tokenisation and Representation How continuous robot actions become discrete tokens an autoregressive transformer can emit, from per-dimension binning to frequency-space compression.
- 02 Evaluating Robot Policies Why a reported 70 percent success rate on a robot task usually carries an error bar of plus or minus 15 points, and what protocol changes make two policies actually comparable.
- 03 Imitation Learning and Diffusion Policies Why cloning a demonstrator's actions drifts into unseen states, and how generative action models such as diffusion policies and action chunking control the drift.
- 04 Massively Parallel Simulation and Sim-to-Real Locomotion How putting the physics engine on the GPU turned weeks of legged-robot RL into minutes, and why domain randomisation is the only reason those policies survive contact with a real floor.
- 05 Sim-to-Real Transfer Why robot policies are trained in simulation, why they break on real hardware, and how domain randomisation closes the reality gap by making the real world look like one more random draw.
- 06 Cross-Embodiment Transfer Why robot data has never had an ImageNet, how Open X-Embodiment pooled 22 robot types into one training set, and what has to be true for data from one arm to help a different arm.
- 07 Flow-Matching Action Experts Why binning continuous robot actions into text tokens caps control frequency, and how attaching a flow-matching action head to a vision-language model reaches 50 Hz dexterous control.
- 08 RT-2 and Web-Scale Robot Learning RT-2 co-trains one transformer on internet vision-language data and robot trajectories by encoding actions as text tokens, transferring semantic web knowledge into robotic control.
- 09 Vision-Language-Action Models Turning a pretrained vision-language model into a robot policy that maps camera images plus a language instruction to motor actions, so the robot inherits web-scale semantic knowledge it could never learn from robot data alone.
- 10 World Models for Control How learning a latent dynamics model lets an agent train inside its own imagination, why that is the answer to robotics' sample-efficiency problem, and where the learned simulator's errors get exploited.
AI for Science AlphaFold, protein language models, materials discovery, and the pitfalls of ML-for-science. 12 concepts · 137 cards
- 01 Machine Learning for Materials Discovery How graph neural networks screen millions of candidate crystals for stability and how machine-learning interatomic potentials approximate DFT cheaply enough to simulate them, plus why an in-silico "stable" material is not yet a real one.
- 02 Pitfalls in ML-for-Science The failure modes, chiefly data leakage, that make machine-learning results in scientific papers look stronger than they replicate, and the reporting standards proposed to catch them.
- 03 Protein Language Models How masked-language-model pretraining over amino-acid sequences produces structure and function signal, and how ESMFold trades some accuracy for dropping the MSA search that AlphaFold2 depends on.
- 04 Self-Driving Labs and Autonomous Experimentation The A-Lab synthesised 41 of 58 target compounds in 17 days with no human intervention, and the dispute that followed — ending in a 2026 author correction — is the clearest available lesson in what an autonomous laboratory actually automates.
- 05 AI for Formal Mathematics How proof assistants turn mathematics into a verifiable reward signal, what AlphaGeometry and AlphaProof achieved at the IMO, and why autoformalisation remains the bottleneck.
- 06 AlphaFold2 and the Protein-Folding Problem How a deep-learning system read co-evolution signal out of aligned protein sequences to predict 3D structure at near-experimental accuracy, and what it still cannot do.
- 07 AlphaFold3 and Biomolecular Co-Folding How AlphaFold3 dropped the protein-only structure module for a diffusion head that denoises raw atoms, letting one model co-fold proteins with ligands, nucleic acids, ions, and modified residues, and how the open reimplementations caught up.
- 08 Generative Protein and Molecule Design Structure prediction reads nature's proteins; generative design writes new ones — how RFdiffusion denoises backbones into existence, why a separate network is needed to choose the sequence, and why the only benchmark that counts is a wet-lab success rate.
- 09 Machine-Learned Interatomic Potentials How equivariant graph networks reach near-quantum accuracy at a fraction of the cost, why symmetry is designed in rather than learned, and what breaks when a potential leaves its training chemistry.
- 10 Neural Operators and PDE Surrogates Why learning a mapping between function spaces is different from fitting a network to a grid, how the Fourier neural operator achieves resolution invariance, and what a surrogate cannot promise.
- 11 Neural Weather Prediction How graph and transformer models trained on reanalysis data overtook physics-based forecasting on most verification targets, what they still depend on, and where the learned approach genuinely fails.
- 12 Single-Cell Foundation Models Pretraining a transformer on tens of millions of single-cell transcriptomes produced scGPT and Geneformer, and then a zero-shot benchmark found both losing to highly variable gene selection — a case study in what "foundation model" does and does not transfer.
09
Classical ML & Statistical Learning
The statistics and non-neural models that still decide most production predictions.
6tracks
43concepts
885cards
5.4hreading
Statistical Inference Estimators, likelihood, the bootstrap, hypothesis testing, multiplicity and what a confidence interval really claims. 7 concepts · 149 cards
- 01 Confidence Intervals and Coverage What the 95% in a 95% interval refers to, why the Wald interval for a proportion is badly behaved near zero, and how a confidence interval differs from the credible interval people usually think they are reading.
- 02 Estimators, Bias, Variance and Consistency What makes one estimator better than another, why the mean squared error splits cleanly into bias squared plus variance, and why an unbiased estimator is frequently the wrong thing to want.
- 03 Hypothesis Tests, Error Types and the Neyman–Pearson Lemma A hypothesis test is a decision rule with two error rates that trade against each other, and the Neyman–Pearson lemma says the likelihood ratio is the rule that buys the most power for a fixed false-positive rate.
- 04 P-Values, Multiplicity and the Garden of Forking Paths What a p-value actually claims, why running twenty tests guarantees a false discovery, and the difference between controlling the family-wise error rate and controlling the false discovery rate.
- 05 Sampling Distributions and the Central Limit Theorem Every standard error and every normal-theory interval is a statement about a distribution you never observe, and the central limit theorem is a promise about that distribution whose speed of delivery depends on skewness, tails and dependence.
- 06 The Bootstrap and Resampling Resampling the data you have to simulate the sampling distribution you never observed, why the percentile interval is not always the right one, and the specific statistics for which the bootstrap silently fails.
- 07 Likelihood Ratio, Wald and Score Tests Three asymptotically equivalent ways to test a parameter from the log-likelihood, which measure the vertical drop, the horizontal distance and the slope at the null, and which can disagree wildly in exactly the samples where the answer matters.
Classical Supervised Learning Linear and logistic regression, regularisation, margins, kernels, and the geometry underneath them. 8 concepts · 156 cards
- 01 Linear Regression as Projection Least squares is orthogonal projection onto the column space of the design matrix, which explains the normal equations, the meaning of residuals, and why nobody who ships numerical code inverts the matrix.
- 02 Logistic Regression and the Log-Odds Why classification is modelled on the log-odds scale rather than the probability scale, why there is no closed-form solution, and what perfect separation does to the coefficients.
- 03 k-Nearest Neighbours and the Curse of Dimensionality The simplest classifier there is comes with a strong guarantee in low dimensions and a quiet collapse in high ones, because neighbourhoods stop being local and distances stop discriminating.
- 04 Cross-Validation and Model Selection Cross-validation estimates how well a learning procedure generalises, not how well your fitted model will, and the moment you use it to pick among models its estimate of the winner is optimistically biased.
- 05 Naive Bayes and Generative Classifiers Naive Bayes models how each class generates its features under an assumption that is almost always false, and still classifies well, because getting the ranking right is far easier than getting the probabilities right.
- 06 Ridge, Lasso and Elastic Net Why an $\ell_1$ penalty produces exact zeros while $\ell_2$ only shrinks, what each does to correlated features, and why regularisation is a bias-variance trade rather than a way to fix a bad model.
- 07 Margins, Kernels and the Support Vector Machine How maximising the distance to the nearest point gives a classifier that depends on a handful of examples, and how the dual formulation lets you work in an infinite-dimensional feature space without ever visiting it.
- 08 Probability Calibration: Platt Scaling and Isotonic Regression A classifier can rank perfectly and still report probabilities that are wrong by a factor of three, and post-hoc calibration fixes the numbers without touching the ranking, at a cost in held-out data that depends on which method you choose.
Trees & Ensembles Impurity splitting, bagging, random forests, gradient boosting, and the engineering inside XGBoost and LightGBM. 7 concepts · 145 cards
- 01 Decision Trees and Impurity Splitting How a greedy search over axis-aligned splits builds a piecewise-constant function, why Gini and entropy almost never disagree, and the specific structural biases that make a single tree unstable.
- 02 Bagging and Random Forests Why averaging unstable models reduces variance, why bootstrap sampling alone is not enough, and what the extra feature subsampling in a random forest is actually buying.
- 03 Stacking and Blending Stacking learns how to combine several models from their out-of-fold predictions, and nearly all of its value and all of its danger come from how those predictions are produced.
- 04 Why Trees Still Beat Deep Nets on Tabular Data The three inductive biases that separate tree ensembles from neural networks on tabular problems, and the specific dataset conditions under which the ordering has been observed to flip.
- 05 Gradient Boosting as Functional Gradient Descent Boosting is gradient descent in function space, where each new tree approximates the negative gradient of the loss, which is what lets a single algorithm fit any differentiable objective.
- 06 Histogram Gradient Boosting: The Engineering Inside XGBoost, LightGBM and CatBoost The three dominant boosting libraries share one objective and differ in how they search for splits, grow trees and handle leakage, and those engineering choices decide speed, memory and which datasets each one overfits.
- 07 TreeSHAP and Feature Importance Impurity, permutation and SHAP importance answer three different questions about a tree ensemble, each with a known bias, and TreeSHAP made the game-theoretic one exact and fast without settling which question you should be asking.
Unsupervised Learning Clustering, mixture models and EM, PCA and SVD, manifold embeddings, and density estimation. 7 concepts · 145 cards
- 01 K-Means and Its Assumptions Lloyd's algorithm is coordinate descent on a squared-error objective, which explains why it always converges, why it converges to the wrong answer without careful seeding, and the exact cluster shapes it cannot represent.
- 02 Density-Based Clustering with DBSCAN and HDBSCAN DBSCAN defines a cluster as a connected region of high point density and labels everything else as noise, and HDBSCAN removes its single density threshold by building the whole hierarchy of thresholds and keeping the most persistent clusters.
- 03 Hierarchical Clustering and Linkage Agglomerative clustering builds a full tree of merges instead of one partition, and the linkage rule that scores each merge decides whether the tree finds crescents, chains through noise, or reproduces k-means.
- 04 PCA, SVD and Whitening Why maximising retained variance and minimising reconstruction error give the same answer, how the SVD computes it without ever forming a covariance matrix, and what whitening destroys.
- 05 UMAP, t-SNE and What They Distort Neighbour embeddings optimise local neighbourhood preservation and nothing else, which makes cluster separation, cluster size and inter-cluster distance in the resulting picture largely uninterpretable.
- 06 Clustering Validation: Silhouette, Gap, ARI and Choosing k Internal indices like silhouette and the gap statistic score a clustering against its own geometric ideal, external indices like ARI and NMI need chance correction to mean anything, and Kleinberg's impossibility theorem explains why no index can settle what the right clustering is.
- 07 Gaussian Mixtures and the EM Algorithm How treating the cluster label as a latent variable turns an intractable likelihood into two closed-form steps, why the likelihood is guaranteed to increase, and why it is unbounded above.
Bayesian Methods Priors and posteriors, MCMC and HMC, variational inference, Gaussian processes and model comparison. 7 concepts · 145 cards
- 01 Hierarchical Models and Partial Pooling A hierarchical model lets groups borrow strength from each other by learning how different they are, which turns the choice between separate estimates and one pooled estimate into a parameter the data decides.
- 02 Posterior Predictive Checks and the Bayesian Workflow A posterior predictive check simulates replicated data from the fitted model and asks whether they look like the real data in the ways that matter, and it is one step in a workflow that also checks priors, computation and the model's ability to recover known parameters.
- 03 Priors, Conjugacy and the Posterior How a prior functions as pseudo-data, why conjugate families make the posterior a closed-form update, and why "uninformative" priors are informative on some scale.
- 04 Bayesian Model Comparison: Bayes Factors, PSIS-LOO and WAIC Bayes factors compare models by how well their priors predicted the data and are acutely sensitive to prior width, while PSIS-LOO and WAIC estimate out-of-sample predictive accuracy from the posterior and come with diagnostics that say when to distrust them.
- 05 Gaussian Processes Placing a prior over functions rather than parameters, which gives exact posterior uncertainty that grows away from the data, at a cubic cost that determines where they are usable.
- 06 MCMC and Hamiltonian Monte Carlo Why sampling from an unnormalised posterior is possible at all, why random-walk proposals fail in high dimensions, and how using gradient information turns a random walk into directed motion.
- 07 Variational Inference and the ELBO Turning integration into optimisation by fitting a tractable distribution to the posterior, and the specific bias that comes from minimising the reverse KL divergence.
Feature Engineering Encoding, missingness, selection, target leakage, and the train-serve skew that eats offline gains. 7 concepts · 145 cards
- 01 Feature Scaling and Transformations Standardisation, power transforms and quantile maps change what distance, penalties and gradients see, so they matter enormously for some model families and not at all for others, and transforming the target quietly changes what the model predicts.
- 02 Categorical Encoding Why one-hot encoding breaks down at high cardinality, how target encoding trades that for a leakage risk it must then defend against, and what each choice assumes about unseen categories.
- 03 Feature Selection: Filter, Wrapper and Embedded Methods Filter, wrapper and embedded methods trade cost against how much of the model they consult, and all three produce wildly optimistic accuracy if the selection step happens outside the cross-validation loop.
- 04 Missing Data Mechanisms Whether imputation is safe depends on why the value is absent, and the three-way distinction between MCAR, MAR and MNAR decides which methods are valid and which quietly bias the result.
- 05 Target Leakage The failure mode where a feature encodes information unavailable at prediction time, why cross-validation cannot detect it, and the three structural forms it takes.
- 06 Train-Serve Skew The gap between how a feature is computed in a training pipeline and how it is computed in a serving path, which silently degrades a model that was never wrong in offline evaluation.
- 07 Class Imbalance, Resampling and Reweighting Rebalancing classes by oversampling, SMOTE or class weights mostly moves the decision threshold while destroying probability calibration, and the evidence suggests strong learners gain little from it compared with choosing the threshold from actual costs.
10
Causal Inference & Experimentation
Telling correlation from cause, and measuring whether a change actually helped.
4tracks
28concepts
557cards
3.6hreading
Causal Foundations Potential outcomes, structural causal models, DAGs, confounding, colliders and the do-operator. 7 concepts · 137 cards
- 01 DAGs, Confounders and Colliders A causal graph turns "which variables should I control for" into a question with a mechanical answer, and shows why conditioning on the wrong variable creates bias rather than removing it.
- 02 Identification Assumptions: Exchangeability, Positivity and Consistency The three conditions that let an observed conditional mean stand in for an unobserved potential outcome, how each one enters the identification proof, and which of them data can and cannot check.
- 03 Potential Outcomes and the Fundamental Problem A causal effect is a comparison of two outcomes for the same unit, only one of which is ever observed, which makes causal inference a missing-data problem rather than a modelling problem.
- 04 Simpson's Paradox and Choosing an Adjustment Set The same data can show an effect in every subgroup and the opposite effect in aggregate, and the arithmetic cannot tell you which is right; only the causal structure can.
- 05 Mediation Analysis: Direct and Indirect Effects Splitting a total effect into the part that runs through a mediator and the part that does not requires counterfactuals that mix two worlds, which is why the decomposition needs assumptions that even a randomised experiment cannot deliver.
- 06 Structural Causal Models and Counterfactuals A structural causal model assigns every variable a mechanism and an exogenous noise term, which is exactly the extra structure needed to answer "what would have happened to this unit", the question neither correlations nor interventions can answer.
- 07 The Do-Operator and Identification The distinction between conditioning on what you observed and intervening to set a value, and why identification is a question about the graph that must be settled before any estimation.
Online Experimentation Power, peeking, sample ratio mismatch, variance reduction, interference and long-term effects. 7 concepts · 142 cards
- 01 Overall Evaluation Criterion and Guardrail Metrics Choosing the metric an experiment is judged on, why the obvious business metrics make bad ones, and how guardrail metrics with non-inferiority tests protect everything the goal metric does not measure.
- 02 Sample Ratio Mismatch When the observed traffic split differs from the intended one, the randomisation is compromised and the effect estimate should be discarded rather than adjusted, because the cause is almost always a mechanism that also biases the metric.
- 03 Statistical Power and the Minimum Detectable Effect Sample size is decided by the smallest effect worth detecting, not by convention, and the fourth-power relationship between effect size and required traffic is why most product experiments are underpowered.
- 04 Interference and Network Effects in Experiments When one unit's treatment affects another unit's outcome, individual randomisation measures a quantity that is neither the treatment effect nor zero, and the standard designs trade bias against a large loss of power.
- 05 Novelty, Primacy and Long-Term Effects Treatment effects that grow or decay as users learn, how to detect and measure that learning, and the two ways to estimate a long-term effect without waiting years, long-running holdouts and surrogate indices.
- 06 Peeking and Sequential Testing Fixed-sample p-values assume the sample size was chosen in advance, so continuously monitoring a dashboard and stopping at significance can inflate the false-positive rate several-fold.
- 07 Variance Reduction with CUPED and Stratification Regressing out pre-experiment behaviour removes variance that has nothing to do with the treatment, buying sensitivity without extra traffic, and the size of the gain is set by one correlation.
Observational Causal Methods Propensity scores, instrumental variables, difference-in-differences, regression discontinuity and synthetic control. 7 concepts · 137 cards
- 01 Difference-in-Differences and Parallel Trends Using a control group's change over time to estimate what the treated group's change would have been, and the untestable assumption that carries the entire argument.
- 02 G-Computation and Regression Adjustment Turning an outcome model into a causal effect by predicting every unit's outcome under each treatment and averaging, and why that is not the same as reading a coefficient off the regression.
- 03 Sensitivity Analysis and E-Values Asking how strong an unmeasured confounder would have to be to explain away an observational result, and the three standard ways to quantify that, from Rosenbaum's gamma to the E-value to partial R-squared bounds.
- 04 Double Machine Learning Using arbitrary machine-learning models to adjust for high-dimensional confounders while keeping a root-n, asymptotically normal estimate of the causal parameter, through orthogonal scores and cross-fitting.
- 05 Instrumental Variables Using a source of variation that affects treatment but has no other path to the outcome, which recovers a causal effect despite unmeasured confounding, for a subpopulation you cannot identify.
- 06 Propensity Scores and Matching Reducing a high-dimensional covariate vector to a single probability of treatment, which makes balancing tractable but does nothing about the confounders you did not measure.
- 07 Regression Discontinuity and Synthetic Control Two designs that manufacture a credible counterfactual, one from an arbitrary threshold in an assignment rule and one from a weighted combination of untreated units.
Policy Learning & Off-Policy Evaluation Heterogeneous treatment effects, uplift modelling, importance sampling estimators and doubly robust methods. 7 concepts · 141 cards
- 01 Counterfactual Risk Minimisation Learning a new policy directly from logged bandit feedback by minimising an importance-weighted risk plus a variance penalty, and why naive IPS objectives overfit to the logging policy's choices rather than to the rewards.
- 02 Doubly Robust Estimation Combining an outcome model with a propensity model so that the estimate stays consistent if either one is correct, and why the modern version adds cross-fitting to make that guarantee usable.
- 03 Heterogeneous Treatment Effects and Uplift Estimating who benefits rather than whether the average benefits, and why the target quantity is never observed for any individual, which breaks every standard model-selection habit.
- 04 Inverse Propensity Scoring for Off-Policy Evaluation Estimating how a new policy would have performed using only logs from an old one, by reweighting each logged decision by how much more likely the new policy was to make it.
- 05 Off-Policy Evaluation for Rankers and LLM Systems Applying counterfactual estimation where the action is a ranked list or a generated response, where the action space is effectively unbounded and the logging policy was never stochastic.
- 06 Self-Normalised and Clipped IPS Estimators The three standard repairs to inverse propensity scoring, self-normalisation, weight clipping and the switch estimator, what bias each buys its variance reduction with, and how to choose between them.
- 07 Slate Off-Policy Evaluation Evaluating ranked lists and page layouts from logs when the number of possible slates is astronomically large, by assuming structure in how slate rewards decompose, with the pseudoinverse estimator as the central example.
11
Time Series & Forecasting
Data with an arrow of time, where shuffling the rows destroys the problem.
4tracks
32concepts
377cards
3.8hreading
Time Series Foundations Stationarity, autocorrelation, ARIMA and state space models, seasonality and honest backtesting. 7 concepts · 133 cards
- 01 Seasonality and Decomposition Splitting a series into trend, seasonal and remainder components, and the choice between additive and multiplicative structure that determines whether the seasonal pattern grows with the level.
- 02 Autocorrelation and ARIMA Reading the autocorrelation and partial autocorrelation functions to identify how much of a series is explained by its own past, and what the AR, I and MA components each represent.
- 03 Backtesting and Temporal Validation Random cross-validation on time series lets a model learn from the future, and the alternatives all trade honesty against how much of the data can be used.
- 04 Exponential Smoothing and the ETS State-Space Form How a one-line recursive average became a family of likelihood-based models with prediction intervals and automatic selection, and why that family remains the benchmark sophisticated forecasters most often fail to beat.
- 05 Stationarity and Differencing Almost every time series method assumes the statistical properties do not change over time, and the transformations that enforce that assumption also change what the model is predicting.
- 06 Vector Autoregression and Granger Causality How a VAR models several series as jointly driven by their own pasts, what a Granger causality test actually establishes, and the specific ways predictive precedence differs from causation.
- 07 The Kalman Filter and Linear Gaussian State-Space Models How two alternating steps, predict and update, compute the exact posterior of a hidden state in a linear Gaussian model, why the same machinery yields the likelihood and a smoother, and what breaks when the model is wrong.
Forecasting at Scale Hierarchical reconciliation, probabilistic forecasts, global models and time-series foundation models. 12 concepts · 105 cards
- 01 Direct versus Recursive Multi-Horizon Forecasting The two ways to turn a one-step model into a 28-step forecast, why theory prefers direct estimation and practice keeps preferring recursion, and the hybrid strategies that sit between them.
- 02 Exogenous Covariates and Known-Future Inputs The three-way split between static attributes, covariates known into the future and covariates observed only up to now, why conflating them creates a silent look-ahead bug, and what each one is worth.
- 03 Forecast Combination and the Diversity Benefit Why averaging forecasts has beaten selecting the best one for fifty years, the variance algebra that explains it, and the combination puzzle where estimated optimal weights lose to the simple mean.
- 04 Global Models Versus Local Models Why fitting one model across thousands of series beats fitting one model per series, what cross-learning provides that per-series estimation cannot, and where local models still win.
- 05 Gradient-Boosted Trees for Forecasting How reframing a forecast as tabular regression over lag and calendar features made LightGBM the default winner of large-scale forecasting competitions, and the three things that framing cannot do.
- 06 Intermittent Demand Forecasting Why series that are mostly zeros defeat ordinary smoothing and ordinary error metrics, how Croston, SBA and TSB separate demand size from demand occurrence, and what the M5 competition showed about where machine learning helps and where it does not.
- 07 Cold-Start Forecasting for New Series What to predict for a series with no history, why attribute-based pooling and pretrained priors are two answers to the same question, and how to evaluate a regime you cannot backtest in the usual way.
- 08 Deep Forecasting Architectures: DeepAR, N-BEATS and TFT Three neural forecasting designs that each made a different bet (autoregressive likelihoods, deep residual stacks, and attention over typed covariates), what each is good for, and the evidence that simple baselines still beat many deep models.
- 09 Forecast Evaluation at Scale Why MAPE fails on the series that matter most, which scale-free metrics work instead, and how aggregating errors across a heterogeneous population hides the failures worth finding.
- 10 Hierarchical Forecasting and Reconciliation Why independently forecasting every level of a hierarchy produces numbers that do not add up, what reconciliation does about it, and why the optimal method also improves accuracy.
- 11 Probabilistic Forecasts and Quantile Loss Why a point forecast is insufficient for any decision involving asymmetric costs, how pinball loss trains quantiles directly, and what makes a distributional forecast well calibrated.
- 12 Time Series Foundation Models What a pretrained forecasting model transfers, why zero-shot forecasting is plausible at all, and how to evaluate the claim against a well-tuned classical baseline.
Anomaly & Changepoint Detection Residual-based detection, CUSUM, seasonal-hybrid methods, and the base-rate problem in alerting. 7 concepts · 72 cards
- 01 The Base Rate Problem in Alerting Why a detector with excellent sensitivity and specificity still produces mostly false alarms, the arithmetic that shows it, and the design responses that make an alerting system usable.
- 02 Bayesian Online Changepoint Detection How tracking a posterior over the time since the last change turns changepoint detection into exact, recursive Bayesian inference, what the run-length recursion computes at each step, and where its assumptions give way.
- 03 CUSUM and Sequential Change Detection Why detecting a small persistent shift needs accumulated evidence rather than a threshold, how the cumulative sum statistic works, and the tradeoff between detection delay and false alarm rate.
- 04 Evaluating Anomaly Detectors Why precision and recall computed per timestamp mislead on ranges, the point-adjustment protocol that inflated a decade of results, and what an honest evaluation reports.
- 05 Multivariate Time Series Anomaly Detection Why anomalies in systems of many sensors often live in broken relationships rather than in any single channel, the classical and deep detectors built to find them, and why the benchmark evidence for the deep ones is much weaker than their reported scores suggest.
- 06 Seasonal-Hybrid Decomposition for Anomalies Why anomaly detection on a seasonal series must remove the seasonality first, how robust decomposition avoids letting anomalies distort the fit, and the residual tests that follow.
- 07 Unsupervised Anomaly Detection Methods The three families of detector that need no labels, what notion of "anomalous" each encodes, and why the choice is a modelling assumption rather than a performance question.
Online & Streaming Learning Incremental updates, concept drift, regret bounds, and models that must learn from a moving distribution. 6 concepts · 67 cards
- 01 Concept Drift and Adaptive Windows How a streaming learner decides which past data still applies, why a fixed window is wrong in both directions, and the detectors that size the window from the data.
- 02 Incremental Model Updates in Production What it takes to update a deployed model continuously, why the appeal is usually smaller than it appears, and the failure modes that make scheduled retraining the safer default.
- 03 Online Gradient Descent and FTRL-Proximal How projected online gradient descent earns its square-root regret, why follow-the-regularised-leader is the same algorithm written lazily, and how that rewrite let Google train sparse click-through models on billions of features.
- 04 Online Learning and Regret Bounds What it means to learn without assuming a data distribution, why regret against the best fixed predictor is the achievable guarantee, and what online convex optimisation delivers.
- 05 Prequential Evaluation Why a stream has no test set, how test-then-train uses every example twice without leaking, and the forgetting mechanisms that make the resulting accuracy curve informative.
- 06 Streaming Algorithms and Sketches How to compute statistics over a stream too large to store, the three canonical sketches and their guarantees, and the mergeability property that makes them work in distributed systems.
12
Graphs, Recommenders & Structured Data
Learning over relations, catalogues and columns rather than free text.
4tracks
26concepts
326cards
3.2hreading
Graph Neural Networks Message passing, expressive power and the WL test, over-smoothing, over-squashing and sampling at scale. 7 concepts · 74 cards
- 01 Expressive Power and the WL Test The exact ceiling on what a message-passing network can distinguish, why the aggregation function determines whether that ceiling is reached, and the graphs it provably cannot tell apart.
- 02 Graph Attention Networks and Graph Transformers How attention replaces fixed neighbour weights in GAT, why the original GAT can only compute a static ranking of neighbours, and how graph transformers attend globally while injecting structure through positional and structural encodings.
- 03 Heterogeneous and Temporal Graphs Why real graphs have typed nodes and edges that a homogeneous model averages together, how relation-specific message passing preserves the distinction, and what adding time requires.
- 04 Link Prediction and Negative Sampling Why predicting edges is not ordinary classification, how negative sampling choices determine what the model learns, and the evaluation leakage that makes published numbers hard to trust.
- 05 Over-Smoothing and Over-Squashing The two distinct reasons deep graph networks fail, why they pull in opposite directions, and the interventions that address each.
- 06 Sampling and Scaling to Large Graphs Why full-batch training does not scale past a certain graph size, the neighbourhood explosion that makes naive mini-batching worse, and the three sampling strategies that resolve it.
- 07 Spectral Graph Convolutions Defining convolution on a graph through the eigenvectors of its Laplacian, how Chebyshev polynomials make the filter local and cheap, and how the familiar GCN layer falls out as a first-order approximation with a renormalisation trick.
Knowledge Graphs Triples and ontologies, entity resolution, embedding-based link prediction, and grounding LLMs in structure. 6 concepts · 65 cards
- 01 Graph Query Languages: SPARQL, Cypher and GQL How SPARQL matches triple patterns over RDF and Cypher matches ASCII-art patterns over property graphs, why path semantics decide whether a query terminates, and where the ISO GQL standard and SQL/PGQ fit as of September 2026.
- 02 Triples, Ontologies and the Open World What the subject-predicate-object model buys, why the open world assumption changes what absence means, and where an ontology's constraints help and hurt.
- 03 Building a Graph from Unstructured Text The extraction pipeline from documents to triples, why open extraction produces an unusable graph, and the schema and validation decisions that determine whether the result is queryable.
- 04 Entity Resolution at Graph Scale Why matching records is quadratic and how blocking makes it tractable, the transitivity trap in clustering matches, and why this determines a knowledge graph's quality more than anything downstream.
- 05 Grounding Language Models in Structured Knowledge What a knowledge graph provides that vector retrieval cannot, the two integration patterns, and where the translation from language to structure fails.
- 06 Knowledge Graph Embeddings How representing entities and relations as vectors turns completion into a scoring problem, what each scoring function can and cannot express, and why evaluation in this area has been unreliable.
Recommender Systems Matrix factorisation, two-tower retrieval, ranking objectives, feedback loops and cold start. 7 concepts · 120 cards
- 01 Cold Start and Content-Based Hybrids Why a new user or item is invisible to collaborative filtering, how hybrid models build representations from metadata instead of interaction history, and the exploration cost that no content feature removes.
- 02 Matrix Factorisation and Implicit Feedback Learning low-rank user and item vectors from a sparse interaction matrix, and why the shift from ratings to clicks changes the loss, the negatives and the meaning of the output.
- 03 Deep CTR Ranking Models: Wide & Deep, DCN and DLRM How ranking models for click-through prediction combine huge sparse embedding tables with explicit feature-interaction layers, what Wide & Deep, Deep & Cross and DLRM each assume about crosses, and why careful benchmarking shrank the differences between them.
- 04 Feedback Loops and Filter Bubbles A recommender trained on data it generated is optimising against its own past choices, which narrows what users see and makes offline evaluation systematically agree with the incumbent.
- 05 Offline Evaluation and Sampled Metrics in Recommenders Why ranking against 100 sampled negatives can reverse which recommender looks best, how random splits leak the future into training, and what a decade of reproducibility studies says about reported progress.
- 06 Sequential Recommendation: SASRec and BERT4Rec Treating a user's history as an ordered sequence and predicting the next item with a transformer, and why the famous SASRec versus BERT4Rec comparison turned out to be about loss functions and training budgets rather than attention direction.
- 07 Two-Tower Retrieval and Candidate Generation Splitting the model so that item representations can be precomputed and searched with approximate nearest neighbours, which is what makes recommending from a hundred-million-item catalogue possible at all.
Tabular Deep Learning Why trees still win, attention over columns, prior-fitted networks and the benchmarks that decide the argument. 6 concepts · 67 cards
- 01 Attention Over Columns How transformer architectures are adapted to tables, what per-feature embeddings buy, and why the attention mechanism addresses the feature selection problem specifically.
- 02 Benchmarking Tabular Methods Honestly Why tabular comparisons disagree so persistently, the tuning asymmetry that produces most of the disagreement, and what a comparison needs to support its conclusion.
- 03 Deep Learning for High-Cardinality Categoricals The one tabular regime where networks clearly win, why embeddings handle millions of levels that trees cannot, and the engineering that dominates such systems.
- 04 Numerical Feature Embeddings for Tabular Models Why feeding a raw scalar into an MLP handicaps it on tabular data, how piecewise linear encodings and trainable periodic features turn each number into a vector, and what Gorishniy et al. (2022) measured when they did.
- 05 Prior-Fitted Networks and In-Context Tabular Learning How a model trained entirely on synthetic datasets can classify a real one without fitting, what approximating the Bayesian posterior in a forward pass means, and the constraints that follow.
- 06 The Inductive Bias Mismatch on Tabular Data The three properties of tabular data that neural networks handle badly and trees handle naturally, why this is a bias mismatch rather than a capacity problem, and what it implies for architecture design.
13
Generative Modelling Beyond Transformers
Diffusion, flows, adversarial games and state space models, and the theory that connects them.
5tracks
37concepts
388cards
4.7hreading
Diffusion Models Forward noising, denoising objectives, samplers, classifier-free guidance and latent diffusion. 7 concepts · 82 cards
- 01 Classifier-Free Guidance The extrapolation trick that made text-to-image work, why it is not sampling from any distribution the model learned, and the fidelity-diversity tradeoff it exposes as a single tunable number.
- 02 Forward Diffusion and Noise Schedules The fixed corruption process that makes diffusion training possible, why the closed form in alpha-bar removes the need to simulate it, and how the schedule silently decides which frequencies the model learns.
- 03 Latent Diffusion and the Autoencoder Bottleneck Why running diffusion in a compressed latent space cuts cost by more than an order of magnitude, what the autoencoder throws away permanently, and the failure modes that belong to the VAE rather than the diffusion model.
- 04 Consistency Models and Few-Step Sampling How consistency models learn a direct map from any point on a probability-flow trajectory to its origin, the difference between distilling that map from a diffusion teacher and training it from scratch, and how latent and adversarial variants reach one to four steps.
- 05 Denoising Parameterisations: Epsilon, x-zero and v Three algebraically equivalent things a diffusion network can predict, why they train to completely different models, and how the choice interacts with the noise schedule and with distillation.
- 06 Diffusion Transformers (DiT) Why replacing the diffusion U-Net with a plain vision transformer over latent patches made image quality a predictable function of forward-pass compute, and how adaLN-Zero conditioning and the MM-DiT joint-attention block made it work.
- 07 Samplers: DDPM, DDIM and Higher-Order Solvers Why sampling is numerical integration of an ODE or SDE rather than a fixed algorithm, how DDIM makes the process deterministic and skippable, and what higher-order solvers buy at the cost of stability.
Variational & Flow Models The ELBO, reparameterisation, normalising flows, flow matching and rectified transport. 6 concepts · 71 cards
- 01 The Reparameterisation Trick Why you cannot backpropagate through a sampling operation, how moving the randomness to an input fixes it, and the variance argument that explains why this beats the score-function estimator.
- 02 Continuous Normalising Flows and Neural ODEs How defining a flow as the solution of an ODE replaces the Jacobian determinant with a trace, why Hutchinson's estimator makes that trace linear in dimension, what the adjoint method buys and costs, and why simulation in the training loop kept these models slow.
- 03 Flow Matching and Conditional Vector Fields How to train a continuous-time generative model by regressing a velocity field without ever simulating an ODE, and why conditioning on a single data point makes an intractable target tractable.
- 04 Normalising Flows and the Cost of Invertibility How change of variables turns a simple density into a complex one with an exact likelihood, why the Jacobian determinant constrains every architectural choice, and what that constraint costs relative to VAEs and diffusion.
- 05 Posterior Collapse and the KL Term The failure where a VAE's latent code carries no information, why a powerful decoder makes it the optimal solution rather than an accident, and the fixes that each buy something different.
- 06 Rectified Flow and Trajectory Straightening Why the generative ODE learned from independent noise-data pairs is curved, how reflow iteratively straightens it, and what each round of straightening costs in fidelity.
Adversarial Generative Models The minimax game, mode collapse, Wasserstein critics, and what FID does and does not measure. 7 concepts · 76 cards
- 01 Adversarial Losses as a Component Why GANs largely lost as standalone generative models but their loss survived inside autoencoders, vocoders and super-resolution, and what a patch discriminator adds that L2 cannot.
- 02 Conditional GANs and Image-to-Image Translation How conditioning information enters a GAN's generator and discriminator, why pix2pix needs paired data while CycleGAN substitutes cycle consistency, what that substitute silently permits, and why the projection discriminator beat concatenation.
- 03 The GAN Minimax Game What the discriminator is actually estimating, why the theoretically clean generator loss cannot be used in practice, and what it means that training seeks an equilibrium rather than a minimum.
- 04 What FID Measures and What It Misses The Gaussian assumption inside Frechet Inception Distance, the sample-size bias that makes numbers incomparable across papers, and why precision and recall metrics exist.
- 05 Mode Collapse and Training Instability Why a GAN generator has no incentive to cover the data distribution, the difference between full and partial collapse, and what each of the standard mitigations actually changes.
- 06 StyleGAN and Style-Based Generation How StyleGAN moved the latent code out of the generator's input and into per-layer modulation through a learned mapping network, and how StyleGAN2 and StyleGAN3 traced visible artefacts back to normalisation, progressive growing and aliasing.
- 07 Wasserstein Critics and the Lipschitz Constraint Why earth mover distance still has a gradient when JS divergence does not, how the Kantorovich duality turns it into a trainable critic, and why enforcing the Lipschitz bound is where every practical difficulty lives.
State Space Models S4, Mamba, selective scan, the recurrence-convolution duality, and where linear-time sequence models pay off. 11 concepts · 94 cards
- 01 Hybrid Attention-SSM Architectures Why the strongest linear-time models are not pure state space models, what a small number of full attention layers restores, and how to reason about the mixing ratio.
- 02 The Fixed State Budget at Inference Why a state space model decodes in constant memory per sequence, how to do the arithmetic against a KV cache, and the operational consequences that constant memory buys and costs.
- 03 Discretising Continuous State Spaces How a continuous-time state space layer becomes a recurrence you can run on tokens, why zero-order hold and the bilinear transform disagree at large step sizes, and why the step size ends up acting as a learned timescale and a gate.
- 04 Distilling Transformers into Linear-Time Models How a pretrained transformer's attention projections are reused to initialise state space layers, why the conversion is staged rather than end to end, and what the resulting hybrids do and do not recover.
- 05 HiPPO and Structured State Initialisation Why a randomly initialised state space layer fails on long sequences, what optimal polynomial projection of history gives you instead, and how the structure that makes it work also makes it computable.
- 06 Long Convolution Models and Implicit Filters How H3 and Hyena reached subquadratic sequence mixing through FFT convolutions with implicitly parameterised filters, what data-controlled gating contributed, and why selectivity displaced the whole family.
- 07 Parallel Scan and Hardware-Aware SSM Kernels How an associative scan recovers training parallelism after selectivity destroys the convolution, and why the arithmetic-intensity argument means the kernel is the architecture.
- 08 Selectivity and Input-Dependent Parameters What linear time-invariant models fundamentally cannot do, how making the state space parameters functions of the input fixes it, and the computational bill that change immediately creates.
- 09 State Capacity and Associative Recall The information-theoretic reason a fixed-size recurrent state cannot copy arbitrary context, what MQAR measures that perplexity hides, and why the gap shows up on exactly the tasks production systems care about.
- 10 Structured State Space Duality The equivalence between selective state space models and a masked form of linear attention, why writing the recurrence as a semiseparable matrix recovers tensor-core throughput, and what the chunked algorithm actually computes.
- 11 The Recurrence-Convolution Duality Why a linear state space layer can be run as a parallel convolution during training and as a constant-memory recurrence at inference, and what the word "linear" is buying.
Energy-Based & Score Models Unnormalised densities, score matching, Langevin dynamics, and the SDE view that unifies the generative families. 6 concepts · 65 cards
- 01 Contrastive Divergence and the Negative Phase How energy-based training approximates an intractable expectation with a short MCMC chain, what bias that introduces, and why the same positive-negative structure appears in contrastive learning and reward modelling.
- 02 Energy-Based Models and the Partition Function The most flexible way to specify a probability distribution, why the normalising constant makes it untrainable by direct likelihood, and the three escape routes that define the rest of the field.
- 03 Langevin Dynamics for Sampling How a noisy gradient ascent on log density becomes a valid sampler, why the noise term is what separates sampling from optimisation, and the mixing failure that makes it impractical alone.
- 04 Noise-Contrastive Estimation How training a classifier to tell data from known noise recovers an unnormalised density and its normalising constant, why the choice of noise decides whether that works, and how word2vec's negative sampling and InfoNCE descend from it.
- 05 Score Matching and Its Denoising Form How targeting the gradient of log density eliminates the partition function, why the naive form requires an intractable Hessian trace, and how adding noise makes the objective a simple regression.
- 06 The SDE View of Generative Models The continuous-time framework in which diffusion, score matching and denoising are the same object, why every SDE has a deterministic twin with identical marginals, and what the unification actually buys.
14
Efficiency, Compression & Edge AI
Making a model smaller, cheaper and local without giving away the thing that made it useful.
5tracks
31concepts
350cards
3.7hreading
Quantisation Post-training and quantisation-aware methods, outlier channels, GPTQ and AWQ, and low-bit arithmetic formats. 6 concepts · 71 cards
- 01 Quantisation Grids, Scale and Zero Point The affine map between floating point and integers, why granularity is the single most consequential choice, and how clipping and rounding errors trade against each other.
- 02 AWQ and Activation-Aware Scaling The observation that weight importance is determined by activation magnitude rather than weight magnitude, and how a per-channel rescaling protects the important weights without keeping any of them in higher precision.
- 03 GPTQ and Second-Order Weight Rounding Why rounding each weight to its nearest grid point is not the best rounding, how the Hessian of the layer reconstruction error tells you what to do instead, and what the approximations cost.
- 04 Low-Bit Number Formats and Microscaling Why floating-point formats at 8 bits and below split differently between exponent and mantissa, what a shared block exponent buys, and how hardware support decides which format wins regardless of its numerical merits.
- 05 Mixed-Precision Bit Allocation and Sensitivity Why a fixed bit budget is better spent unevenly across layers, how Hessian-based sensitivity from HAWQ turns an exponential search into an integer programme, and where the second-order proxy and the hardware disagree.
- 06 Quantisation-Aware Training and the Straight-Through Estimator How you backpropagate through a step function that has zero gradient everywhere, what QAT buys over post-training methods, and why it is used far less than its accuracy would justify.
Knowledge Distillation Soft targets and temperature, sequence-level and on-policy distillation, and when a student beats its teacher. 6 concepts · 71 cards
- 01 Feature and Attention Transfer in Distillation How FitNets hints, attention transfer, TinyBERT and MiniLM supervise a student's intermediate representations rather than only its outputs, what alignment problem each design has to solve, and the evidence that feature matching can make a student worse.
- 02 Temperature and Dark Knowledge What information a teacher's full output distribution carries that a hard label does not, why temperature is needed to expose it, and the gradient-scaling correction that everyone forgets.
- 03 On-Policy Distillation and Exposure Bias Why a student trained only on teacher trajectories cannot recover from its own errors, how generating from the student fixes the state distribution, and what reverse KL buys and costs.
- 04 Self-Distillation and Born-Again Networks The result that a student identical in architecture to its teacher often outperforms it, the competing explanations for why, and where the effect is genuinely useful rather than merely surprising.
- 05 The Capacity Gap in Distillation Why a stronger teacher can produce a worse student, what the intermediate-teacher fix does, and how to reason about the ratio between teacher and student capacity.
- 06 Token-Level Versus Sequence-Level Distillation Why matching a teacher's per-token distributions is not the same as matching its outputs, and how training on teacher-generated sequences changes the objective from mode-covering to mode-seeking.
Sparsity & Pruning Magnitude and second-order criteria, structured versus unstructured sparsity, and the hardware that rewards it. 6 concepts · 69 cards
- 01 Magnitude Pruning and the Lottery Ticket Hypothesis Why the simplest possible pruning criterion is so hard to beat, what the lottery ticket experiment actually claims, and the rewinding detail that decides whether it reproduces.
- 02 Structured Versus Unstructured Sparsity Why 90 percent unstructured sparsity can be slower than a dense matmul, what removing a whole channel buys instead, and how to decide which side of the tradeoff a deployment sits on.
- 03 Depth Pruning and Layer Removal for LLMs Deleting whole transformer blocks is the one form of structured pruning that speeds up batch-one decoding almost linearly, and the evidence says deep layers are surprisingly removable for knowledge recall and surprisingly necessary for reasoning.
- 04 Dynamic Sparse Training Training a sparse network from scratch by continuously rewiring which weights exist, why the gradients of absent weights are the key signal, and what stops this from replacing dense training.
- 05 N:M Semi-Structured Sparsity The compromise pattern that hardware can accelerate, why 2:4 specifically, and the gap between the theoretical 2x and what a full model actually achieves.
- 06 Second-Order and Activation-Aware Pruning How the Hessian of the loss gives a principled importance score, why the exact version is intractable, and the two approximations that made one-shot pruning of large language models work.
Efficient Architectures Small language models, depth-width tradeoffs, weight sharing, and architectures designed for a latency budget. 6 concepts · 65 cards
- 01 Depthwise Separable Convolutions and Mobile Networks Splitting a convolution into a per-channel spatial filter and a 1x1 channel mixer cuts its arithmetic by nearly an order of magnitude, which built MobileNet and EfficientNet, and also produced the field's clearest case of FLOPs failing to predict speed.
- 02 Small Language Models and the Overtraining Regime Why compute-optimal training is the wrong objective when inference dominates the bill, how far past Chinchilla the good small models actually go, and what stops the trend.
- 03 Weight Tying and Parameter Sharing Where reusing one set of weights in several places is nearly free, where it costs real capability, and why the embedding matrix is the case everyone gets right and the layer stack is the case everyone gets wrong.
- 04 Architecting to a Latency Budget How to design a model backwards from a millisecond target using arithmetic intensity, why parameter count is the wrong currency, and the design moves that actually reduce time to first and subsequent tokens.
- 05 Depth Versus Width Tradeoffs Why two models with identical parameter counts behave differently depending on how the parameters are arranged, what depth buys that width cannot, and how the hardware votes for width.
- 06 Hardware-Aware Architecture Search Why optimising a model for FLOPs produces slow models, how measured latency became the objective instead, and what makes once-for-all supernet training the practical form of the idea.
On-Device & Edge AI Mobile NPUs, memory-bound inference on consumer silicon, compilation targets and privacy-driven local models. 7 concepts · 74 cards
- 01 Cloud-Edge Split Inference Where to cut a model between device and datacenter is an optimisation over per-layer compute, activation size and link conditions; it works well for feed-forward vision models and poorly for autoregressive LLMs, which split at the query instead.
- 02 Compilation Targets and Runtime Fragmentation The path from a trained PyTorch model to something that runs on a phone, why the intermediate format is where most deployment failures happen, and what each of the major runtimes assumes.
- 03 Mobile NPUs and the Accelerator Zoo What a phone's neural processing unit is good at, why the same model runs at wildly different speeds on the CPU, GPU and NPU of one device, and the fallback that silently destroys performance.
- 04 Privacy-Driven Local Inference What running a model locally actually guarantees, which parts of the pipeline still leak, and how hybrid designs preserve most of the property while escalating the hard requests.
- 05 Federated Learning on Edge Devices Training a shared model without collecting the data, why non-IID client distributions break the averaging assumption, and the systems constraints that decide what is actually trainable.
- 06 On-Device LLM Inference Constraints The arithmetic that decides whether a language model can run on a phone, why bandwidth rather than compute is the binding constraint, and what the KV cache does to the memory budget.
- 07 TinyML and Microcontroller Inference On a microcontroller the binding constraint is not compute but two memories, flash for weights and code and SRAM for activations, and the techniques that work (static arenas, peak-memory-aware search, patch-based execution) all attack peak SRAM.
15
Search & Information Retrieval
Thirty years of ranking research that RAG rediscovered, usually the hard way.
5tracks
34concepts
605cards
4.1hreading
Classical Information Retrieval Inverted indexes, TF-IDF and BM25, query processing, and the lexical baselines that refuse to be beaten. 7 concepts · 141 cards
- 01 Text Analysis: Stemming, Lemmatisation and Stopwords How an analyzer chain turns raw text into index terms, what stemming, lemmatisation and stop lists trade between recall and precision, and why these decisions are fixed at indexing time.
- 02 BM25 and Term Frequency Saturation Why a term appearing twenty times should not score ten times higher than one appearing twice, how BM25 encodes that as a saturating function, and what its two parameters actually control.
- 03 Inverted Indexes and Postings Lists The data structure that makes text search sublinear in corpus size, why postings are stored as sorted document IDs, and how gap encoding turns a list of integers into a few bits each.
- 04 TF-IDF and the Vector Space Model Queries and documents as weighted term vectors ranked by cosine, why each weight multiplies local and collection-wide evidence, and why plain cosine normalisation systematically favours short documents.
- 05 Dynamic Pruning with WAND and Block-Max Retrieving the exact top-k without scoring most of the candidates, by maintaining an upper bound on what each document could score and skipping everything that cannot beat the current threshold.
- 06 Learned Sparse Retrieval Using a language model to assign weights over the vocabulary, including terms not present in the text, so semantic matching runs on an inverted index instead of a vector index.
- 07 Query Likelihood Language Models for Retrieval Ranking documents by the probability that each document's language model generates the query, and why the smoothing that makes this work quietly reintroduces IDF weighting and length normalisation.
Learning to Rank Pointwise, pairwise and listwise objectives, LambdaMART, position bias and counterfactual training. 7 concepts · 133 cards
- 01 LTR Datasets and Ranking Feature Design What the standard learning-to-rank benchmarks (LETOR, MSLR-WEB30K, Yahoo, Istella) actually contain, how their hand-built feature vectors are designed, and what their results do and do not tell you about a production ranker.
- 02 Multi-Stage Ranking Cascades Why production search and recommendation rank in stages of increasing cost over shrinking candidate sets, how to budget candidates and latency per stage, and why the first stage caps what every later stage can achieve.
- 03 Pointwise, Pairwise and Listwise Objectives Ranking is not regression, and the three families of learning-to-rank losses differ in how much of the ranking structure they put inside the objective rather than leaving to a sort.
- 04 Click Models: Cascade, DBN and UBM Generative models of how users scan and click a result page, which let a search log be turned into relevance estimates by modelling what each user probably examined and whether a click satisfied them.
- 05 Counterfactual Learning to Rank Training a ranker on logged clicks while correcting for the bias in how those clicks were generated, which turns a biased log into an unbiased estimate of a ranking objective.
- 06 LambdaRank and LambdaMART The trick of defining a gradient without ever defining a loss, which lets gradient boosting optimise a discontinuous ranking metric directly, and why the result dominated learning to rank for a decade.
- 07 Position Bias and the Examination Hypothesis Clicks measure relevance multiplied by the chance the user looked, so training on raw clicks teaches a ranker to reproduce whatever ranking generated the logs.
Query Understanding Intent classification, spelling and segmentation, expansion, rewriting, and conversational query resolution. 7 concepts · 133 cards
- 01 Query Autocompletion Proposing full queries from a typed prefix within the gap between two keystrokes, from the most-popular-completion baseline and compressed top-k tries to personalised and generative completion, and the harms of suggesting things people did not ask for.
- 02 Query Entity Recognition and Linking Finding the entity mentions in a two- or three-word query and resolving each to a knowledge-base identifier, under a latency budget measured in milliseconds and with almost no context to disambiguate from.
- 03 Query Intent and Taxonomies The same string can be three different requests, and classifying which one determines whether the right answer is a document, an entity, an action, or a generated response.
- 04 Spelling Correction and Query Segmentation Fixing what the user typed before matching it, where the hard part is not generating candidates but deciding whether the original was wrong at all.
- 05 Conversational Query Rewriting Turning a context-dependent follow-up into a self-contained query, which is what lets a stateless retriever serve a stateful conversation.
- 06 Parsing Queries into Structured Filters Turning a free-text query into attribute constraints and residual text that a structured index can execute exactly, from CRF query tagging to schema-constrained LLM parsing, and why negation is the case that makes parsing worth its cost.
- 07 Query Expansion and Pseudo-Relevance Feedback Adding terms to a query to bridge vocabulary mismatch, and the drift failure that occurs when the terms are harvested from results that were wrong to begin with.
Search Evaluation Pooling and judgments, nDCG and MRR, interleaving, online metrics, and why offline gains vanish online. 7 concepts · 129 cards
- 01 Precision, Recall and Average Precision The binary-relevance metric family that underpins ranked retrieval evaluation, from set precision and recall through P@k, R-precision and interpolated curves to average precision, and the user model each one quietly assumes.
- 02 Test Collections, Pooling and Judgment Bias The Cranfield paradigm made retrieval a measurable science, and the pooling shortcut that makes it affordable quietly penalises any system unlike the ones that built the pool.
- 03 nDCG, MRR and Graded Relevance The main ranking metrics differ in what they assume about the user, and choosing one is choosing a model of how far someone reads and what they are looking for.
- 04 Interleaving and Online Evaluation Mixing two rankers' results into a single list and attributing clicks gives a within-user paired comparison that detects differences far faster than an A/B test on the same traffic.
- 05 LLM Relevance Judgments Using large language models to label query-document relevance for offline evaluation, what the agreement numbers actually show, and the circularity that appears once the systems being judged use the same kind of model as the judge.
- 06 Significance Testing in IR Evaluation How to decide whether a MAP or nDCG difference over a few dozen topics is signal, which paired tests actually hold their error rates on IR data, and why testing many variants at once quietly inflates what counts as significant.
- 07 Why Offline Gains Vanish Online The recurring experience that an offline nDCG improvement produces no measurable online effect, and the four distinct mechanisms that cause it.
Vector Databases Index families, filtered search, freshness and deletes, sharding, and the operational reality of billion-scale ANN. 6 concepts · 69 cards
- 01 Operating a Vector Store in Production The operational facts that decide whether a vector search system works, why memory is the binding constraint, and the migration nobody plans for.
- 02 Choosing an Index Family The four index families and the workload property that selects each, why the recall-latency-memory triangle admits no universal answer, and how to read a published benchmark.
- 03 Disk-Based ANN Search and DiskANN How DiskANN serves a billion vectors from one machine with 64 GB of RAM by keeping compressed codes in memory and a Vamana graph on SSD, why the number of disk round trips is the budget that shapes every design choice, and how the Fresh and Filtered variants extend it.
- 04 Filtered Vector Search Why combining a metadata filter with a nearest-neighbour search is harder than either alone, the two naive strategies and how each fails, and what a native implementation does instead.
- 05 Freshness, Deletes and Index Maintenance Why approximate indexes are built for static data, what insertion and deletion do to their structure over time, and the segment architecture that reconciles freshness with query performance.
- 06 Sharding and Distributed Vector Search The two ways to partition a vector collection across machines, why one preserves recall and the other preserves latency, and the fan-out arithmetic that decides tail latency.
16
Data & Feature Engineering
The pipelines, formats and contracts that decide whether a model ever sees correct inputs.
5tracks
37concepts
381cards
4.5hreading
Data Modelling & Storage Columnar formats, table formats and the lakehouse, partitioning, and modelling choices that decide query cost. 11 concepts · 95 cards
- 01 Arrow and Zero-Copy Columnar Interchange The difference between a storage format and a memory format, why Arrow eliminated a conversion step that cost more than the query, and where copies still happen despite the name.
- 02 Columnar Formats and Why They Win What changes when values of the same column sit next to each other on disk, why that makes compression an order of magnitude better, and the workloads where row storage is still correct.
- 03 Compaction and the Small File Problem Why streaming ingestion destroys query performance on a table that is otherwise perfectly healthy, what compaction actually costs, and the write-amplification trap in naive compaction policies.
- 04 Copy-on-Write versus Merge-on-Read The two ways to change one row inside an immutable file, why the choice is really a decision about where to spend amplification, and what deletion vectors changed.
- 05 Dimensional Modelling and the Case for Wide Tables Why star schemas were designed for a storage economics that no longer holds, what they still provide, and how to decide between a normalised model and one wide denormalised table.
- 06 Object Storage Semantics for Analytics Why a lakehouse built on S3 behaves nothing like one built on a filesystem, and the four properties of object storage that decide table layout, commit protocols and query cost.
- 07 Slowly Changing Dimensions and History How SCD types 1, 2, 3 and 6 decide what a dimension remembers when an attribute changes, why bitemporal modelling separates when something was true from when you learned it, and how an overwrite quietly leaks the future into a training set.
- 08 Table Formats and the Lakehouse What a directory of Parquet files cannot do, how a metadata layer adds atomic commits and time travel on top of immutable object storage, and where the abstraction leaks.
- 09 Partitioning, Clustering and File Sizing The three physical layout decisions that determine query cost, why partitioning on a high-cardinality column is the classic disaster, and how to reason about the right file size.
- 10 Predicate Pushdown and Column Statistics How a query engine avoids reading data it can prove is irrelevant, the hierarchy of pruning from partition to page, and why statistics that exist can still be useless.
- 11 Storage Layout for ML Training Reads Why the access pattern of a training loop is the one pattern the lakehouse was not designed for, and how sharded sequential formats trade shuffle quality for two orders of magnitude fewer requests.
Batch & Streaming Pipelines Event time versus processing time, watermarks, exactly-once semantics, backfills and orchestration. 7 concepts · 74 cards
- 01 Backfills and Reprocessing Why recomputing history is a different operation from running a pipeline fast, the resource and correctness hazards specific to it, and the design choices that make a pipeline backfillable at all.
- 02 Change Data Capture and the Outbox Pattern Why writing to a database and a message broker from the same request cannot be made atomic, how log-based change data capture turns the database's own commit log into the event stream, and how the outbox pattern keeps that stream a contract rather than a leak of internal schema.
- 03 Event Time Versus Processing Time The two clocks every streaming system has, why using the wrong one produces results that change on replay, and the skew that makes correctness a latency tradeoff.
- 04 Orchestration, Dependencies and Idempotency What an orchestrator is actually for, why retries are the whole reason tasks must be idempotent, and the difference between scheduling on time and scheduling on data availability.
- 05 Stream Windowing: Tumbling, Sliding and Session How a windowing function assigns each event on an unbounded stream to finite groups, why the three standard shapes differ in state cost and in whether their boundaries are shared across keys, and how late data makes session windows merge after results have been emitted.
- 06 Exactly-Once Semantics, Precisely What "exactly once" actually guarantees, why it is a statement about effects rather than about deliveries, and the two mechanisms that provide it.
- 07 Watermarks and Allowed Lateness The heuristic that lets a streaming system decide a window is complete, why it is always wrong in one of two directions, and how triggers and lateness policies turn a single answer into a sequence of refinements.
Feature Stores Offline-online parity, point-in-time correctness, materialisation, and the failure they exist to prevent. 6 concepts · 71 cards
- 01 Feature Reuse, Discovery and Ownership The organisational argument for a feature store, why reuse is harder than it sounds, and the governance problems that appear once several teams depend on one definition.
- 02 Materialisation and the Online Store How features get from an analytical table into a millisecond-latency lookup, the freshness-cost tradeoff each materialisation strategy makes, and why the online store's data model is nothing like the offline one.
- 03 Embeddings as Managed Features Why an embedding stored in a feature store is only meaningful alongside the exact encoder that produced it, what an encoder upgrade costs in backfill and downstream retraining, and how backward-compatible training and learned alignment trade model quality for avoiding that cost.
- 04 Offline-Online Parity and Training-Serving Skew Why the same feature computed by two pipelines is rarely the same number, the categories of divergence, and the architectural choices that eliminate rather than manage the problem.
- 05 Point-in-Time Correctness The join that a feature store exists to get right, why a naive join on entity ID leaks the future into training labels, and what an as-of join costs to compute.
- 06 Streaming Aggregations for Real-Time Features How to compute a sliding-window count over millions of entities within a serving latency budget, why exact windows are usually unaffordable, and the approximations that are safe.
Data Quality & Contracts Expectations and assertions, schema evolution, producer-consumer contracts, and detecting silent corruption. 6 concepts · 69 cards
- 01 Data Contracts as Producer Obligations Why moving the schema definition to the producer changes the economics of data quality, what a contract must contain beyond field types, and the organisational conditions under which contracts actually work.
- 02 Expectations, Assertions and Where to Put Them The categories of check a data pipeline can make, why the placement of a check matters more than the check itself, and the failure mode of testing only what is easy to test.
- 03 Testing Data Transformations How to test the logic of a SQL or dataframe transformation rather than the data flowing through it, using fixture-based unit tests, property-based tests that state invariants, and reconciliation checks that compare what went in with what came out.
- 04 Anomaly Detection on Data Pipelines Why learned thresholds beat static ones for volume and freshness, the base-rate problem that makes naive alerting useless, and how to structure alerts so that people still read them after six months.
- 05 Detecting Silent Corruption The failures that produce structurally valid, plausible, wrong data, why per-row checks cannot find them, and the reconciliation techniques that can.
- 06 Schema Evolution and Compatibility Modes The precise definitions of backward, forward and full compatibility, why the direction depends on whether you deploy producers or consumers first, and the changes that are always breaking.
Data Governance & Lineage Catalogues, column-level lineage, retention and deletion, access control and provenance for training corpora. 7 concepts · 72 cards
- 01 Catalogues, Metadata and Discovery The three kinds of metadata a catalogue holds, why the technical layer is the only one that stays accurate for free, and what makes a catalogue get used rather than abandoned.
- 02 Column-Level Lineage What lineage answers that a dependency graph cannot, why column granularity changes the questions you can ask, and the parsing problem that makes it hard to capture correctly.
- 03 Data Classification and Sensitivity Labelling Why a sensitivity scheme is only as good as how labels propagate through joins, aggregates, embeddings and models, how the high-water-mark rule makes propagation mechanical, and the over-classification and inference failures that quietly turn labels into noise.
- 04 Access Control for Analytical Data Why role-based access breaks down on a data platform, how attribute-based policies and dynamic masking replace it, and the derived-table hole that undermines both.
- 05 Consent and Purpose Limitation for Training Data Why data collected for one purpose cannot simply be reused to train a model under GDPR, how the Article 6(4) compatibility test and the legitimate-interest route work in practice, what EDPB Opinion 28/2024 and the courts have said, and where the law stands as of September 2026.
- 06 Provenance for Training Corpora Why the governance question for training data is harder than for analytical data, what a defensible provenance record contains, and the specific obligations that make it necessary.
- 07 Retention, Deletion and the Right to Erasure Why deleting a row is not deleting data, the specific places copies survive in a modern data platform, and the architectures that make erasure tractable.
17
MLOps & Platform Engineering
Everything between a notebook that works and a system that keeps working.
6tracks
37concepts
419cards
4.5hreading
Experiment Tracking & Reproducibility Run metadata, seeds and determinism, environment capture, and what it takes to rebuild a result a year later. 6 concepts · 67 cards
- 01 Environment Capture and Pinning Why a requirements file does not describe an environment, the layers below Python that also move, and the tradeoff between reproducibility and being able to patch a vulnerability.
- 02 Run Metadata and What to Record The minimum set of facts that makes a training run comparable and rebuildable a year later, why metrics are the least important part, and the discipline that makes tracking survive contact with a deadline.
- 03 Data Versioning and Content Addressing Why versioning datasets is harder than versioning code, how content addressing makes it tractable without copying, and what a dataset version has to mean to be useful.
- 04 Hyperparameter Search Hygiene Why random search beats grid search, how early-stopping schedulers change the budget calculation, and the selection bias that makes the best run's reported score an overestimate.
- 05 Multi-Seed Reporting and Run-to-Run Variance How many seeds a comparison needs, which sources of variance a seed sweep should randomise, and the reporting conventions that stop a one-run improvement from being mistaken for a result.
- 06 Seeds and the Limits of Determinism Why setting a seed does not make a training run reproducible, the specific sources of non-determinism on a GPU, and how to decide how much determinism is worth paying for.
Model Registry & Versioning Artefact lineage, promotion gates, rollback, and versioning models and their data together. 6 concepts · 67 cards
- 01 Model Cards and Intended Use What documentation a model version needs to be safely reused by someone who did not build it, why disaggregated evaluation is the substantive part, and how model cards become checkbox exercises.
- 02 The Registry as a Promotion Gate Why a model registry is a workflow rather than a storage location, what a stage transition should require, and the difference between a registry that records decisions and one that enforces them.
- 03 Model Lineage Across Retraining Why a chain of fine-tuned and distilled models makes provenance a graph rather than a record, what has to be tracked at each edge, and the failures that follow from losing it.
- 04 Rollback, Pinning and Reproducible Serving What it takes to revert a model change under incident conditions, why the model is usually not the only thing that has to revert, and the mechanisms that make a rollback fast rather than heroic.
- 05 Versioning Compound LLM Systems Why an LLM application's version must bind the model snapshot, prompt, retrieval index, tools and decoding settings into one identifier, and how provider deprecations turn rollback from a button into a migration.
- 06 What a Model Version Actually Contains Why weights alone are not a deployable model, the full set of artefacts that must move together, and the coupling failures that occur when one of them is versioned separately.
CI/CD for ML Testing pipelines that emit models, data-dependent tests, progressive delivery and automated retraining. 6 concepts · 69 cards
- 01 Build Once, Promote the Artefact Why rebuilding per environment reintroduces every difference the pipeline was meant to eliminate, how configuration is separated from the immutable artefact, and where ML deployments break the pattern.
- 02 Testing Pipelines That Emit Models What can be tested deterministically in an ML codebase, why model quality is not a unit test, and the layered test strategy that keeps CI fast while still catching the failures that matter.
- 03 Automated Retraining and Its Triggers When a retraining loop is worth building, the four trigger designs and what each optimises, and the failure modes that make automated retraining actively dangerous.
- 04 Data-Dependent Tests and Behavioural Suites Why aggregate metrics are a poor gate, how curated behavioural cases catch regressions that averages hide, and the discipline of turning every production failure into a permanent test.
- 05 LLM Regression Testing in CI How to gate pull requests on a stochastic system, covering golden-set sizing, paired comparisons, judge variance, flaky thresholds and the per-PR cost budget that decides what runs where.
- 06 Progressive Delivery for Models The staged rollout patterns that separate correctness validation from quality validation, why shadow deployment answers a question canary cannot, and what each stage should be watching.
ML Observability & Drift Feature and prediction monitoring, delayed labels, drift statistics, and alerting that does not cry wolf. 6 concepts · 71 cards
- 01 Prediction Logging and Traceability What to record at inference so that a question asked three months later has an answer, why the feature vector matters more than the input, and the sampling and privacy tradeoffs.
- 02 The Three Drifts and How They Differ Covariate shift, label shift and concept drift decomposed precisely, why only one of them necessarily degrades a model, and which of them your monitoring can actually see.
- 03 Drift Statistics and What They Miss PSI, KL divergence, KS and MMD compared on what they detect and where they fail, why per-feature tests miss joint shifts, and the multiple-comparison problem that makes wide monitoring noisy.
- 04 Monitoring Without Labels What to watch when ground truth arrives months late or never, why prediction distributions and confidence are the highest-value proxies, and how to estimate performance from unlabelled data.
- 05 Observability for LLM Applications Why the classical monitoring stack does not transfer to systems with free-text output, what a trace over an agent must capture, and the online quality signals that work without ground truth.
- 06 Slice-Based Monitoring and Alert Design Why aggregate model metrics hide failures on important subgroups, how sample size and multiple testing limit what per-slice monitoring can detect, and how to route slice alerts so they stay actionable.
GPU Fleet & Capacity Scheduling and quota, fragmentation, preemption, multi-tenancy, and planning capacity under lumpy demand. 7 concepts · 76 cards
- 01 Capacity Planning Under Lumpy Demand Why GPU demand does not smooth the way CPU demand does, how to reason about the reserved-versus-on-demand mix, and the lead times that make this a quarters-ahead decision.
- 02 Fault Tolerance for Long Training Runs Why failure is the expected case at scale, the arithmetic that sets checkpoint frequency, and the detection problem that makes silent corruption worse than a crash.
- 03 GPU Health Checks and Silent Data Corruption What the published interruption statistics from Llama 3, Gemini and Meta's research clusters say about hardware failure at scale, and how burn-in, periodic checks, straggler detection and SDC screening keep bad nodes out of synchronous jobs.
- 04 Gang Scheduling and Fragmentation Why a distributed training job cannot start until every worker starts, how that requirement produces both deadlock and stranded capacity, and the placement constraints that make a cluster's usable size smaller than its size.
- 05 Multi-Tenancy and GPU Sharing Why one job per GPU wastes most of the hardware for small models, the three sharing mechanisms and their isolation guarantees, and the interference that makes sharing unsuitable for latency-sensitive work.
- 06 Preemption, Priority and Spot Capacity How interruptible capacity changes the cost of training by a large factor, what a job must be able to do to use it, and the failure modes that make spot capacity a false economy for the unprepared.
- 07 Topology-Aware Scheduling How NVLink domains, rail-optimised fabrics and switch hierarchy make GPU placement a performance variable, the collective cost model that quantifies it, and how schedulers trade placement quality against queueing delay.
Cost & FinOps for AI Unit economics per request, token accounting, reserved versus spot capacity, and attributing spend to features. 6 concepts · 69 cards
- 01 Reserved, On-Demand and the Shape of Commitment How to choose a commitment mix when demand is uncertain, why the break-even is simply a price ratio, and the option value that makes shorter commitments rational despite costing more.
- 02 Self-Hosting Versus API Break-Even A worked break-even model for serving an open-weights model yourself against paying per token, showing why utilisation, not the hourly GPU price, usually decides the answer, and which assumptions flip it.
- 03 Spend Guardrails and Quotas Why AI spend can rise by orders of magnitude in hours, the layered controls that bound it without blocking legitimate work, and the design of a kill switch that is actually usable.
- 04 Unit Economics of an AI Feature How to build a cost-per-request figure that survives scrutiny, why the marginal cost of an LLM feature does not fall with scale the way software's does, and the retry and failure multipliers everyone forgets.
- 05 Token Accounting and Cost Attribution Why a single provider invoice cannot be allocated to teams, features or customers without instrumentation, and the tagging discipline that makes AI spend attributable.
- 06 Training Cost Estimation Before You Commit The FLOP arithmetic that converts a model and dataset size into a GPU-hour figure, why achieved utilisation is the term that decides the answer, and the overheads that turn an estimate into a budget.
18
Security, Privacy & Adversarial ML
Attacks on models, data and the supply chain, and the defences that survive contact.
5tracks
32concepts
349cards
3.9hreading
Adversarial Robustness Perturbation attacks, adversarial training, certified defences, and the robustness-accuracy tradeoff. 6 concepts · 65 cards
- 01 Adversarial Examples and the Threat Model Why an imperceptible perturbation flips a confident classifier, what a norm ball is actually assuming, and why stating the threat model precisely is the first substantive step in any robustness claim.
- 02 Adversarial Training and Certified Defences The min-max formulation behind the only empirical defence that has held up, what a certificate actually guarantees, and the gap between certified and empirical robustness.
- 03 Black-Box and Transfer Attacks How an attacker with only query access, or none, still produces adversarial inputs by estimating gradients from scores, walking the decision boundary from labels, or crafting on a surrogate and relying on transfer, and why the same logic carries GCG suffixes from open-weight models to closed LLMs.
- 04 Data Poisoning and Backdoors How an attacker who controls a small fraction of training data can install a trigger, why backdoors are nearly invisible to standard evaluation, and what the web-scale training setting makes possible.
- 05 Model Extraction and Membership Inference What an attacker can learn from query access alone, why confident outputs leak training set membership, and the defences that trade utility for protection.
- 06 Robustness Evaluation That Means Something Why fixed-attack benchmarks systematically overstate robustness, what an adaptive evaluation requires, and the reporting practices that make a robustness claim checkable.
Privacy-Preserving ML Differential privacy accounting, federated learning, secure aggregation, and the utility cost of each guarantee. 7 concepts · 76 cards
- 01 PII Detection and Redaction Pipelines Why regular expressions catch the easy half, how the recall-utility tradeoff differs between training corpora and live traffic, and the re-identification risk that survives field-level redaction.
- 02 k-Anonymity and Re-identification Why removing names does not anonymise a table, what k-anonymity guarantees and the attacks it does not stop, how the Netflix Prize data was de-anonymised from a handful of ratings, and why differential privacy replaced syntactic anonymity as the formal standard.
- 03 Differential Privacy and the Epsilon You Actually Get What the differential privacy guarantee says precisely, how DP-SGD achieves it through clipping and noise, and why the epsilon values used in practice permit far more leakage than the formalism's reputation suggests.
- 04 Federated Learning and Secure Aggregation Why keeping data on device is not by itself a privacy guarantee, how gradient inversion recovers training inputs, and what secure aggregation does and does not prevent.
- 05 Machine Unlearning What it would mean to remove a training example's influence from a model, why retraining is the only exact method, and the approximate approaches and their verification problem.
- 06 Secure Multiparty Computation and Homomorphic Encryption for ML How secret sharing and homomorphic encryption let a server run a model on inputs it cannot read, why non-linear layers dominate the bill, and the measured overheads, from seconds for ResNet-50 to minutes per token for a 7B LLM, that decide where cryptographic inference is actually deployed.
- 07 Synthetic Data and What It Does Not Protect Why generating data from a model trained on real records does not by itself anonymise anything, the utility-privacy frontier that no generator escapes, and the evaluation that a synthetic dataset needs.
LLM Application Security Injection across trust boundaries, tool and sandbox escape, secret exposure and threat modelling for agents. 7 concepts · 74 cards
- 01 Secrets and Credentials in LLM Context Why anything placed in a model's context should be treated as disclosed to whoever can influence its output, how credential brokering keeps secrets out of the prompt entirely, and where secrets arrive uninvited through users, logs and indexed tools.
- 02 Agent Permissions and Blast Radius How to design tool access for a component you must assume is compromised, why capability scoping beats identity-based permissions here, and the reversibility test that decides what needs confirmation.
- 03 Output Handling and Downstream Injection Why model output is untrusted input to everything it touches, the injection classes that follow from rendering or executing it, and the encoding discipline that prevents them.
- 04 Prompt Injection as a Trust Boundary Failure Why injection is architectural rather than a prompting problem, the distinction between direct and indirect injection that decides severity, and why no known technique closes it.
- 05 RAG Poisoning and Retrieval Trust How a handful of crafted passages in a corpus of millions can control what a RAG system says, why detection-based defences have fallen short, and why the durable controls are provenance on the write path and aggregation that bounds any single passage's influence.
- 06 Red-Teaming and Security Evaluation for LLM Apps What automated attack generation covers, why a pass rate is not a security property against an adversary who retries, and how to structure an evaluation that informs a decision.
- 07 Threat Modelling an LLM System The trust boundaries specific to model-based applications, an inventory of assets and adversaries worth enumerating, and how to turn that into controls rather than a document.
Model Provenance & Watermarking Output watermarking, content credentials, fingerprinting weights and detecting extraction. 6 concepts · 65 cards
- 01 Content Credentials and Provenance Metadata The complementary approach of signing assertions about how content was made, why cryptographic provenance is strong where watermarking is weak, and the stripping problem that limits it.
- 02 Disclosure Obligations for Generated Content What the emerging transparency rules actually require, why machine-readable marking and human-visible disclosure are separate obligations, and the design decisions a deployer has to make.
- 03 Detecting Synthetic Media Without Watermarks Why passive detectors work in the lab and fail in deployment, the base rate problem that makes accusation dangerous, and what the evidence supports doing instead.
- 04 Invisible Image Watermarking and Its Robustness How learned image watermarks hide a detectable signal in pixels, in decoder weights or in the initial diffusion noise, how the detection threshold sets the false-positive rate, and why regeneration attacks strip most of them at little cost to image quality.
- 05 Model Fingerprinting and Weight Attribution How to prove a deployed model was derived from yours, the difference between backdoor-style and intrinsic fingerprints, and why fine-tuning is the adversary that matters.
- 06 Text Watermarking and the Detectability Tradeoff How a statistical signal is embedded in generated text by biasing the sampler, the detection test that makes it verifiable, and the reasons the scheme survives paraphrase poorly.
ML Supply Chain Security Untrusted weights and datasets, deserialisation risk, dependency and registry attacks, and signing artefacts. 6 concepts · 69 cards
- 01 Dependency and Registry Attacks in ML Stacks Why ML environments are unusually exposed to package-level attacks, the specific techniques that keep working, and the controls that actually reduce exposure.
- 02 Untrusted Weights and Deserialisation Risk Why loading a model file can execute code, what safetensors changed, and the checks that belong in any pipeline that downloads weights from a public hub.
- 03 Vetting Third-Party Models and Datasets A practical intake process for an artefact you did not produce, what each check can and cannot establish, and how to size the effort to the deployment's exposure.
- 04 Securing Model Serving Infrastructure Why an inference server is an internet-facing distributed system with a large native attack surface, what recent CVEs in Triton, vLLM and Ray teach about where it breaks, and what GPU isolation and confidential computing do and do not protect.
- 05 Securing the Training Pipeline Why the training environment is a high-value target with unusually broad access, the specific credential and isolation failures that recur, and the controls proportionate to what a compromise would yield.
- 06 Signing, Attestation and SBOMs for Models How software supply chain frameworks map onto model artefacts, what a model bill of materials should contain, and why the interesting claims are about the training process rather than the file.
19
Governance, Risk & Responsible AI
Frameworks, regulation and audit evidence, treated as engineering rather than paperwork.
5tracks
33concepts
340cards
4.0hreading
AI Governance Frameworks NIST AI RMF, ISO/IEC 42001, internal review boards, and turning principles into gates that actually block. 7 concepts · 68 cards
- 01 ISO/IEC 42001 and Certifiable Management Systems What a management system standard certifies, why certification is about process consistency rather than model quality, and where it fits alongside a risk framework.
- 02 Risk Tiering and Impact Assessment How to classify AI systems by consequence rather than by technology, what an impact assessment should establish before a system is built, and why the affected-population question is the one that changes designs.
- 03 The NIST AI Risk Management Framework What the four functions of the AI RMF actually ask an organisation to do, why Govern is the one that determines whether the rest happens, and what a voluntary framework can and cannot deliver.
- 04 Third-Party AI Vendor Risk Management Why buying AI moves the risk without moving the accountability, what due diligence can and cannot learn from a model vendor, and the contract and monitoring controls that make a hosted model governable.
- 05 Incident Response for AI Systems Why an AI incident often has no error to page on, what detection has to rely on instead, and the response steps that differ from a conventional outage.
- 06 Model Risk Management from SR 11-7 to SR 26-2 How bank model risk management built validation around effective challenge, why the 2026 rewrite made it principles-based and pushed generative AI out of scope, and how its three validation components still carry over to LLM systems.
- 07 Turning Principles into Gates That Block Why AI principles documents change nothing on their own, the properties a gate needs to have force, and the design that keeps a review board from becoming either a bottleneck or a formality.
AI Regulation & Compliance The EU AI Act risk tiers, sectoral rules, transparency obligations and evidence a regulator will accept. 7 concepts · 66 cards
- 01 The AI Act Compliance Timeline as It Now Stands The phased application dates, the deferral of high-risk obligations agreed in 2026, and what remained on the original schedule when the rest moved.
- 02 The EU AI Act Risk Tiers How the Act classifies systems into prohibited, high-risk, transparency-obligated and minimal, why the classification turns on use rather than technology, and where the boundaries are genuinely unclear.
- 03 US State and Local AI Laws How New York City's bias-audit law, Colorado's rewritten AI Act and California's frontier-model and automated-decision rules create a moving patchwork, and why federal preemption pressure makes the effective dates less stable than the statutes suggest.
- 04 Evidence a Regulator Will Accept The difference between a policy and evidence of its operation, what an assessor actually asks for, and how to instrument a system so compliance artefacts are produced automatically rather than assembled retrospectively.
- 05 General-Purpose AI Model Obligations and the Code of Practice The two-tier regime the EU AI Act applies to general-purpose AI models, how a training-compute number triggers the systemic-risk tier, and what signing the July 2025 Code of Practice does and does not buy a provider now that enforcement has begun.
- 06 Provider and Deployer Obligations Why the same system carries different duties depending on your role, the actions that turn a deployer into a provider, and how the split shapes contracts between the two.
- 07 Sectoral Rules and Overlapping Regimes Why the AI Act is rarely the only regime applying, how data protection, sector rules and product safety interact with it, and the practical approach to satisfying several at once.
Fairness & Bias Group and individual criteria, impossibility results, measurement under missing attributes, and mitigation costs. 6 concepts · 67 cards
- 01 Where Bias Enters the Pipeline The six distinct points at which disparity is introduced, why calling them all "biased data" prevents fixing them, and which stage each mitigation actually addresses.
- 02 Bias in Generative Models Why classification fairness metrics do not transfer to open-ended generation, the harm categories that appear instead, and the evaluation approaches that produce actionable findings.
- 03 Group Fairness Criteria and Why They Conflict Demographic parity, equalised odds and calibration stated precisely, the impossibility result showing you cannot have all three, and what choosing between them commits you to.
- 04 Individual and Counterfactual Fairness Two attempts to define fairness for a person rather than a group, the Lipschitz condition that treats similar people similarly and the causal condition that asks what the decision would have been had the person's group been different, and why each relocates the value judgement into an object someone must build.
- 05 Measuring Fairness Without the Attribute The methods for estimating disparity when group membership is unavailable, why proxy inference introduces error in a direction that matters, and how to report a result built on an estimate.
- 06 Mitigation at Pre-, In- and Post-Processing The three points at which a fairness intervention can act, what each costs in accuracy and in flexibility, and the legal constraint that decides which are available.
Transparency & Documentation Model and system cards, datasheets, disclosure of evaluations, and documentation that survives an audit. 7 concepts · 74 cards
- 01 Datasheets for Datasets The questions a dataset's documentation has to answer for someone deciding whether to use it, why the composition and collection sections carry the weight, and what happens when a dataset outlives the answers.
- 02 Disclosure Without Overclaiming What honest capability communication looks like, why published evaluation numbers mislead by default, and the specific claims that most often outrun their evidence.
- 03 Counterfactual Explanations and Algorithmic Recourse How a counterfactual explanation is computed as a constrained optimisation, why the nearest counterfactual is often the wrong advice once features cause one another, and how recourse breaks when constraints are missing, the model is retrained, or the explanation is gamed.
- 04 Documentation That Survives an Audit The properties that separate documentation an assessor accepts from documentation they discount, why generated beats written, and how to structure a technical file so it stays true as the system changes.
- 05 Explaining a Decision to the Person Affected Why feature attributions are not explanations for a subject, what a counterfactual explanation provides instead, and the gap between technical interpretability and the account a person is owed.
- 06 Local Explanations: LIME, SHAP and Their Limits How the two dominant model-agnostic explainers turn one prediction into feature weights, the axioms that make Shapley values attractive and the sampling that makes KernelSHAP an approximation, and the evidence that both can be unstable, contradictory and deliberately fooled.
- 07 System Cards and Documenting the Whole Pipeline Why documenting a model is insufficient when behaviour is produced by a pipeline, what a system card adds, and how to keep documentation current when the system changes weekly.
AI Assurance & Audit Third-party evaluation, red-team evidence, incident reporting, and control testing for AI systems. 6 concepts · 65 cards
- 01 Assurance for Continuously Changing Systems Why point-in-time assurance is a poor fit for systems that retrain weekly, what continuous assurance requires instead, and how to define the change that resets the conclusion.
- 02 Conformity Assessment and Third-Party Certification Who is allowed to declare an AI system compliant under the EU AI Act, why most high-risk systems are self-assessed while biometrics can need a notified body, how harmonised standards create a presumption of conformity that did not yet exist as of September 2026, and how this differs from ISO/IEC 42001 certification audited under ISO/IEC 42006.
- 03 Control Testing for AI Systems How to test whether a stated control actually operates, the difference between design and operating effectiveness, and the AI-specific controls whose testing is unfamiliar to conventional auditors.
- 04 Independent Evaluation and Structured Access Why external scrutiny requires access that providers have reasons to withhold, the mechanisms proposed to reconcile the two, and what safe harbour would need to cover.
- 05 Red-Team Evidence and Its Limits What a red-team exercise contributes to an assurance case, why coverage cannot be quantified, and how to report results so they inform a decision rather than reassuring the reader.
- 06 What an AI Audit Can and Cannot Establish The three things an audit might mean, why access level determines what conclusions are available, and the gap between certifying a process and certifying an outcome.
20
Human-AI Interaction, Product & Economics
The people using the system, the product decisions around it, and what compute actually costs.
6tracks
37concepts
403cards
4.4hreading
Interaction Design for AI Latency and streaming affordances, error recovery, steering controls, and designing for probabilistic output. 6 concepts · 65 cards
- 01 Citations and Source Attribution in AI Interfaces Why a citation marker in a generated answer is a claim about support that is wrong a measurable fraction of the time, how citation recall and precision quantify that, and why the marker raises trust whether or not it is accurate.
- 02 Designing for Probabilistic Output Why interfaces built on the assumption of correct output fail when output is usually correct, the design moves that make errors survivable, and the cost of hiding uncertainty.
- 03 Latency, Streaming and Perceived Speed Why time to first token dominates perceived speed, how streaming changes what users tolerate, and the interaction costs that come with it.
- 04 Feedback Collection That Is Worth Having Why thumbs-up and thumbs-down produce almost no usable signal, which implicit behaviours carry more information, and how to collect explicit feedback that can actually train something.
- 05 Interfaces for Multi-Step Agents What changes when the system acts over minutes rather than responds in seconds, how to make a long trajectory legible without demanding constant attention, and where the intervention points belong.
- 06 Steering, Correction and Repair What a user does when the output is nearly right, why regeneration is the wrong primary affordance, and the controls that let someone converge rather than resample.
AI Product Management Scoping around uncertainty, quality bars, offline-to-online metric ladders and shipping under model drift. 6 concepts · 65 cards
- 01 Pricing and Packaging an AI Feature Why marginal cost changes the pricing question, the three models in use and what each fails at, and the guardrails a pricing decision needs when usage is heavy-tailed.
- 02 Prototyping AI Features Before the Model Is Ready How Wizard-of-Oz studies, model-in-the-loop prototypes and eval sets written as specifications let a team learn what users need and what the model must do before anyone commits to building it, and the specific ways each method lies.
- 03 Scoping Under Capability Uncertainty Why you cannot specify an AI feature the way you specify software, the cheap experiments that resolve the uncertainty, and the scoping decisions that determine whether a feature is buildable at all.
- 04 Deciding Where the Human Stays The four automation levels available for any decision, the expected-cost calculation that selects between them, and why partial automation is usually right and usually hardest to design.
- 05 Metric Ladders from Offline to Online The chain from a model metric to a business outcome, why each link is weaker than teams assume, and how to validate the links rather than assuming them.
- 06 Shipping Under Model Drift Why an AI product's behaviour changes without a release, what that does to roadmaps and commitments, and the practices that make a product resilient to a dependency that moves on its own.
Trust Calibration & Reliance Over-reliance and automation bias, confidence display, verification cost and human-AI complementarity. 6 concepts · 71 cards
- 01 Appropriate Reliance and Its Two Failures Why the goal is calibrated trust rather than more trust, how over-reliance and under-reliance each destroy the value of a system, and what determines which one a deployment gets.
- 02 Confidence Communication and Calibration What it means for a confidence signal to be calibrated, why models are systematically overconfident, and the forms of communication that help a person rather than a metric.
- 03 Explanations and Their Effect on Reliance Why showing people the reasons behind an AI recommendation often raises acceptance of wrong recommendations as much as right ones, what cognitive forcing and cost-benefit accounts say about the mechanism, and when an explanation actually helps a person catch errors.
- 04 Human-AI Complementarity Why a human-AI team frequently performs worse than the better of its parts, what complementarity requires, and the conditions under which combining actually helps.
- 05 Sycophancy and the Agreement Problem Why preference training produces models that agree with users, the specific behaviours it manifests as, and why it is a trust problem rather than a politeness one.
- 06 The Verification Cost Problem Why an assistant only saves time when checking is cheaper than doing, the tasks where that asymmetry holds and where it inverts, and the design work that creates it.
Human Data & Annotation Guideline design, inter-annotator agreement, preference collection, rater sourcing and label noise. 7 concepts · 74 cards
- 01 Active Learning and Annotation Budgets How choosing which examples to label, by uncertainty or by coverage, can cut annotation cost, why cold starts and sampling bias undermine it, and why the gains that looked large for models trained from scratch often shrink or vanish with pretrained ones.
- 02 Guidelines Are the Model Specification Why the annotation guideline determines what the model learns more than the architecture does, what a usable guideline contains, and the iteration loop that produces one.
- 03 Inter-Annotator Agreement and What It Bounds Why raw agreement overstates reliability, what Cohen's and Krippendorff's coefficients correct for, and why agreement is the ceiling on any model trained from those labels.
- 04 LLM-Assisted Annotation and Label Verification What the evidence says about language models as annotators, why a labeller that is 90% accurate can still produce biased conclusions, and the audit and routing designs that combine cheap model labels with a small human sample into statistically valid results.
- 05 Label Noise and Learning Through It How random and systematic label noise differ in their effect on a model, why memorisation of noisy labels happens late in training, and the techniques that find mislabelled data cheaply.
- 06 Preference Data Collection Why pairwise comparison replaced absolute rating for alignment data, the biases that contaminate it, and the design choices that determine what a reward model actually learns.
- 07 Rater Sourcing, Quality and Welfare The tradeoffs between crowd, vendor, expert and internal annotation, the mechanisms that maintain quality, and the working conditions that shape both the data and the ethics.
Compute Economics Capex versus tokens, utilisation and depreciation, price-performance curves and the cost floor of inference. 6 concepts · 63 cards
- 01 Concentration, Supply and the Compute Market Why AI compute has an unusual supply structure, what the constraints actually are at each layer, and how that shapes strategy for organisations that only want to buy some.
- 02 Energy and Power as the Binding Constraint Why a grid connection measured in megawatts, not a budget measured in dollars, now caps how much AI compute can be built, how PUE converts a power limit into an accelerator count, and why the demand forecasts disagree by a factor of two.
- 03 Training Versus Inference Spend Why inference dominates the lifetime bill for any successful model, the crossover arithmetic, and how that changes which optimisations are worth doing.
- 04 Capex, Depreciation and the Cost of a GPU-Hour How a purchased accelerator's cost becomes an hourly rate, why the depreciation schedule is the contested assumption, and what utilisation does to the answer.
- 05 The Cost Floor of Inference What sets the minimum achievable cost per token, why memory bandwidth rather than compute is the binding constraint, and which techniques move the floor rather than approaching it.
- 06 The Price-Performance Curve Why cost per unit of AI capability has fallen far faster than hardware improvement alone, the three compounding contributions, and what that implies for planning.
AI Diffusion & Labour Adoption measurement, task-level exposure, productivity studies and what the evidence does and does not show. 6 concepts · 65 cards
- 01 Adoption Patterns and What Usage Data Shows What measured usage reveals about where these systems are actually applied, the augmentation-automation split, and why usage concentration differs from exposure predictions.
- 02 Task Exposure Versus Job Replacement Why the unit of analysis is the task rather than the occupation, what exposure measures and what it does not, and the two opposing forces that determine whether exposure becomes displacement.
- 03 General-Purpose Technologies and the Productivity J-Curve Why technologies that eventually transform an economy first depress measured productivity, how unmeasured intangible investment produces a J-shaped path in the statistics, and why the theory is hard to test while you are standing in the dip.
- 04 Measuring Diffusion Honestly Why most claims about AI's economic impact rest on evidence that cannot support them, the hierarchy of evidence quality, and the specific questions to ask of any figure.
- 05 Skill Distribution and Who Benefits The consistent finding that gains concentrate among lower-skilled workers, the competing explanations for it, and the conditions under which the pattern reverses.
- 06 What the Productivity Studies Actually Found The controlled experiments measuring AI's effect on work output, why their results range from large gains to measured slowdowns, and what distinguishes the settings.