Concept library
883 concepts across 20 domains and 101 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
13Generative Modelling Beyond Transformers
14Efficiency, Compression & Edge AI
15Search & Information Retrieval
16Data & Feature Engineering
17MLOps & Platform Engineering
18Security, Privacy & Adversarial ML
19Governance, Risk & Responsible AI
20Human-AI Interaction, Product & Economics
13
Generative Modelling Beyond Transformers
Diffusion, flows, adversarial games and state space models, and the theory that connects them.
5tracks
25concepts
310cards
3.0hreading
Diffusion Models Forward noising, denoising objectives, samplers, classifier-free guidance and latent diffusion. 5 concepts · 68 cards
- 01 Classifier-Free Guidance The extrapolation trick that made text-to-image work, why it is not sampling from any distribution the model learned, and the fidelity-diversity tradeoff it exposes as a single tunable number.
- 02 Forward Diffusion and Noise Schedules The fixed corruption process that makes diffusion training possible, why the closed form in alpha-bar removes the need to simulate it, and how the schedule silently decides which frequencies the model learns.
- 03 Latent Diffusion and the Autoencoder Bottleneck Why running diffusion in a compressed latent space cuts cost by more than an order of magnitude, what the autoencoder throws away permanently, and the failure modes that belong to the VAE rather than the diffusion model.
- 04 Denoising Parameterisations: Epsilon, x-zero and v Three algebraically equivalent things a diffusion network can predict, why they train to completely different models, and how the choice interacts with the noise schedule and with distillation.
- 05 Samplers: DDPM, DDIM and Higher-Order Solvers Why sampling is numerical integration of an ODE or SDE rather than a fixed algorithm, how DDIM makes the process deterministic and skippable, and what higher-order solvers buy at the cost of stability.
Variational & Flow Models The ELBO, reparameterisation, normalising flows, flow matching and rectified transport. 5 concepts · 64 cards
- 01 The Reparameterisation Trick Why you cannot backpropagate through a sampling operation, how moving the randomness to an input fixes it, and the variance argument that explains why this beats the score-function estimator.
- 02 Flow Matching and Conditional Vector Fields How to train a continuous-time generative model by regressing a velocity field without ever simulating an ODE, and why conditioning on a single data point makes an intractable target tractable.
- 03 Normalising Flows and the Cost of Invertibility How change of variables turns a simple density into a complex one with an exact likelihood, why the Jacobian determinant constrains every architectural choice, and what that constraint costs relative to VAEs and diffusion.
- 04 Posterior Collapse and the KL Term The failure where a VAE's latent code carries no information, why a powerful decoder makes it the optimal solution rather than an accident, and the fixes that each buy something different.
- 05 Rectified Flow and Trajectory Straightening Why the generative ODE learned from independent noise-data pairs is curved, how reflow iteratively straightens it, and what each round of straightening costs in fidelity.
Adversarial Generative Models The minimax game, mode collapse, Wasserstein critics, and what FID does and does not measure. 5 concepts · 62 cards
- 01 Adversarial Losses as a Component Why GANs largely lost as standalone generative models but their loss survived inside autoencoders, vocoders and super-resolution, and what a patch discriminator adds that L2 cannot.
- 02 The GAN Minimax Game What the discriminator is actually estimating, why the theoretically clean generator loss cannot be used in practice, and what it means that training seeks an equilibrium rather than a minimum.
- 03 What FID Measures and What It Misses The Gaussian assumption inside Frechet Inception Distance, the sample-size bias that makes numbers incomparable across papers, and why precision and recall metrics exist.
- 04 Mode Collapse and Training Instability Why a GAN generator has no incentive to cover the data distribution, the difference between full and partial collapse, and what each of the standard mitigations actually changes.
- 05 Wasserstein Critics and the Lipschitz Constraint Why earth mover distance still has a gradient when JS divergence does not, how the Kantorovich duality turns it into a trainable critic, and why enforcing the Lipschitz bound is where every practical difficulty lives.
State Space Models S4, Mamba, selective scan, the recurrence-convolution duality, and where linear-time sequence models pay off. 5 concepts · 58 cards
- 01 Hybrid Attention-SSM Architectures Why the strongest linear-time models are not pure state space models, what a small number of full attention layers restores, and how to reason about the mixing ratio.
- 02 HiPPO and Structured State Initialisation Why a randomly initialised state space layer fails on long sequences, what optimal polynomial projection of history gives you instead, and how the structure that makes it work also makes it computable.
- 03 Parallel Scan and Hardware-Aware SSM Kernels How an associative scan recovers training parallelism after selectivity destroys the convolution, and why the arithmetic-intensity argument means the kernel is the architecture.
- 04 Selectivity and Input-Dependent Parameters What linear time-invariant models fundamentally cannot do, how making the state space parameters functions of the input fixes it, and the computational bill that change immediately creates.
- 05 The Recurrence-Convolution Duality Why a linear state space layer can be run as a parallel convolution during training and as a constant-memory recurrence at inference, and what the word "linear" is buying.
Energy-Based & Score Models Unnormalised densities, score matching, Langevin dynamics, and the SDE view that unifies the generative families. 5 concepts · 58 cards
- 01 Contrastive Divergence and the Negative Phase How energy-based training approximates an intractable expectation with a short MCMC chain, what bias that introduces, and why the same positive-negative structure appears in contrastive learning and reward modelling.
- 02 Energy-Based Models and the Partition Function The most flexible way to specify a probability distribution, why the normalising constant makes it untrainable by direct likelihood, and the three escape routes that define the rest of the field.
- 03 Langevin Dynamics for Sampling How a noisy gradient ascent on log density becomes a valid sampler, why the noise term is what separates sampling from optimisation, and the mixing failure that makes it impractical alone.
- 04 Score Matching and Its Denoising Form How targeting the gradient of log density eliminates the partition function, why the naive form requires an intractable Hessian trace, and how adding noise makes the objective a simple regression.
- 05 The SDE View of Generative Models The continuous-time framework in which diffusion, score matching and denoising are the same object, why every SDE has a deterministic twin with identical marginals, and what the unification actually buys.