Concept library
883 concepts across 20 domains and 101 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
13Generative Modelling Beyond Transformers
14Efficiency, Compression & Edge AI
15Search & Information Retrieval
16Data & Feature Engineering
17MLOps & Platform Engineering
18Security, Privacy & Adversarial ML
19Governance, Risk & Responsible AI
20Human-AI Interaction, Product & Economics
14
Efficiency, Compression & Edge AI
Making a model smaller, cheaper and local without giving away the thing that made it useful.
5tracks
25concepts
308cards
3.0hreading
Quantisation Post-training and quantisation-aware methods, outlier channels, GPTQ and AWQ, and low-bit arithmetic formats. 5 concepts · 64 cards
- 01 Quantisation Grids, Scale and Zero Point The affine map between floating point and integers, why granularity is the single most consequential choice, and how clipping and rounding errors trade against each other.
- 02 AWQ and Activation-Aware Scaling The observation that weight importance is determined by activation magnitude rather than weight magnitude, and how a per-channel rescaling protects the important weights without keeping any of them in higher precision.
- 03 GPTQ and Second-Order Weight Rounding Why rounding each weight to its nearest grid point is not the best rounding, how the Hessian of the layer reconstruction error tells you what to do instead, and what the approximations cost.
- 04 Low-Bit Number Formats and Microscaling Why floating-point formats at 8 bits and below split differently between exponent and mantissa, what a shared block exponent buys, and how hardware support decides which format wins regardless of its numerical merits.
- 05 Quantisation-Aware Training and the Straight-Through Estimator How you backpropagate through a step function that has zero gradient everywhere, what QAT buys over post-training methods, and why it is used far less than its accuracy would justify.
Knowledge Distillation Soft targets and temperature, sequence-level and on-policy distillation, and when a student beats its teacher. 5 concepts · 64 cards
- 01 Temperature and Dark Knowledge What information a teacher's full output distribution carries that a hard label does not, why temperature is needed to expose it, and the gradient-scaling correction that everyone forgets.
- 02 On-Policy Distillation and Exposure Bias Why a student trained only on teacher trajectories cannot recover from its own errors, how generating from the student fixes the state distribution, and what reverse KL buys and costs.
- 03 Self-Distillation and Born-Again Networks The result that a student identical in architecture to its teacher often outperforms it, the competing explanations for why, and where the effect is genuinely useful rather than merely surprising.
- 04 The Capacity Gap in Distillation Why a stronger teacher can produce a worse student, what the intermediate-teacher fix does, and how to reason about the ratio between teacher and student capacity.
- 05 Token-Level Versus Sequence-Level Distillation Why matching a teacher's per-token distributions is not the same as matching its outputs, and how training on teacher-generated sequences changes the objective from mode-covering to mode-seeking.
Sparsity & Pruning Magnitude and second-order criteria, structured versus unstructured sparsity, and the hardware that rewards it. 5 concepts · 62 cards
- 01 Magnitude Pruning and the Lottery Ticket Hypothesis Why the simplest possible pruning criterion is so hard to beat, what the lottery ticket experiment actually claims, and the rewinding detail that decides whether it reproduces.
- 02 Structured Versus Unstructured Sparsity Why 90 percent unstructured sparsity can be slower than a dense matmul, what removing a whole channel buys instead, and how to decide which side of the tradeoff a deployment sits on.
- 03 Dynamic Sparse Training Training a sparse network from scratch by continuously rewiring which weights exist, why the gradients of absent weights are the key signal, and what stops this from replacing dense training.
- 04 N:M Semi-Structured Sparsity The compromise pattern that hardware can accelerate, why 2:4 specifically, and the gap between the theoretical 2x and what a full model actually achieves.
- 05 Second-Order and Activation-Aware Pruning How the Hessian of the loss gives a principled importance score, why the exact version is intractable, and the two approximations that made one-shot pruning of large language models work.
Efficient Architectures Small language models, depth-width tradeoffs, weight sharing, and architectures designed for a latency budget. 5 concepts · 58 cards
- 01 Small Language Models and the Overtraining Regime Why compute-optimal training is the wrong objective when inference dominates the bill, how far past Chinchilla the good small models actually go, and what stops the trend.
- 02 Weight Tying and Parameter Sharing Where reusing one set of weights in several places is nearly free, where it costs real capability, and why the embedding matrix is the case everyone gets right and the layer stack is the case everyone gets wrong.
- 03 Architecting to a Latency Budget How to design a model backwards from a millisecond target using arithmetic intensity, why parameter count is the wrong currency, and the design moves that actually reduce time to first and subsequent tokens.
- 04 Depth Versus Width Tradeoffs Why two models with identical parameter counts behave differently depending on how the parameters are arranged, what depth buys that width cannot, and how the hardware votes for width.
- 05 Hardware-Aware Architecture Search Why optimising a model for FLOPs produces slow models, how measured latency became the objective instead, and what makes once-for-all supernet training the practical form of the idea.
On-Device & Edge AI Mobile NPUs, memory-bound inference on consumer silicon, compilation targets and privacy-driven local models. 5 concepts · 60 cards
- 01 Compilation Targets and Runtime Fragmentation The path from a trained PyTorch model to something that runs on a phone, why the intermediate format is where most deployment failures happen, and what each of the major runtimes assumes.
- 02 Mobile NPUs and the Accelerator Zoo What a phone's neural processing unit is good at, why the same model runs at wildly different speeds on the CPU, GPU and NPU of one device, and the fallback that silently destroys performance.
- 03 Privacy-Driven Local Inference What running a model locally actually guarantees, which parts of the pipeline still leak, and how hybrid designs preserve most of the property while escalating the hard requests.
- 04 Federated Learning on Edge Devices Training a shared model without collecting the data, why non-IID client distributions break the averaging assumption, and the systems constraints that decide what is actually trainable.
- 05 On-Device LLM Inference Constraints The arithmetic that decides whether a language model can run on a phone, why bandwidth rather than compute is the binding constraint, and what the KV cache does to the memory budget.