Concept library
1015 concepts across 20 domains and 101 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
13Generative Modelling Beyond Transformers
14Efficiency, Compression & Edge AI
15Search & Information Retrieval
16Data & Feature Engineering
17MLOps & Platform Engineering
18Security, Privacy & Adversarial ML
19Governance, Risk & Responsible AI
20Human-AI Interaction, Product & Economics
07
Reasoning, Evaluation & Safety
Models that think longer, the evals that measure them, and the failure modes that matter.
3tracks
54concepts
748cards
7.3hreading
Reasoning Models Test-time compute, process reward models, the o-series, DeepSeek-R1 and contamination. 15 concepts · 196 cards
- 01 Chain of Thought, Self-consistency, and Search at Inference A tour of the inference-time reasoning toolkit - from zero-shot CoT prompts to MCTS-decoded reasoning trees, and when each pays for itself.
- 02 Hybrid Reasoning Models One set of weights that can answer instantly or think at length, how mode switching and thinking budgets are trained in, and why the industry converged on hybrids over separate reasoning models.
- 03 OpenAI o1, o3 and the Reasoning-Model Family What is publicly known and what is speculated about OpenAI's reasoning line, why the chain of thought is hidden, and what o3's ARC-AGI result actually proved.
- 04 Reasoning Evals and the Contamination Problem A guided tour of the reasoning benchmark canon, why each saturated faster than the field expected, and the move to live held-out evals as the contamination crisis bites.
- 05 Test-time Compute Scaling Why "thinking longer" at inference can substitute for "training bigger", how the trade-off is operationalised as a token budget, and where the strategy stops paying.
- 06 Chain-of-Thought Faithfulness Whether a model's stated reasoning is the reasoning that produced its answer, how the hint-injection test measures it, and why unfaithful CoT undermines monitoring more than it undermines accuracy.
- 07 Controlling Reasoning Length The mechanisms that set how long a model thinks, from s1's crude budget forcing to length-targeted RL, and why overthinking makes length control a quality problem rather than just a cost problem.
- 08 DeepSeek-R1 and the Open Reasoning Recipe How DeepSeek's R1 pipeline produced o1-class reasoning with an open paper, an open model, and a recipe other labs could replicate within weeks.
- 09 Deliberative Alignment Teaching a model the text of its own safety policy and training it to reason over that text before answering, which improves jailbreak robustness and overrefusal at the same time.
- 10 Inference-Time Scaling Laws and Thinking Budgets How accuracy trades against tokens spent at inference, why the optimal strategy depends on question difficulty, and what a thinking budget actually buys.
- 11 Process Reward Models and Verifiable Rewards Why scoring every step of a reasoning trace beats scoring only the final answer, and how Ai2 and DeepSeek replaced PRMs entirely with programmatic correctness checks.
- 12 Reasoning Distillation and Budget Forcing A thousand carefully chosen reasoning traces and a decoding trick can move a 32B model past o1-preview on competition maths, which says something uncomfortable about what reasoning training is actually teaching.
- 13 The Limits of Self-Correction Why asking a model to review its own answer often makes it worse, what separates intrinsic from extrinsic correction, and the oracle-label leak that inflated the early results.
- 14 Tool-Integrated Reasoning Interleaving code execution with natural-language reasoning removes an entire class of error the model cannot fix by thinking harder, and creates a new class of error it cannot see.
- 15 Verifier-Guided Search and Best-of-N Sampling many candidate solutions is only useful if you can pick the right one, and the gap between what sampling can reach and what selection can find is the central constraint on test-time scaling.
Evaluation & MLOps Benchmarks, LLM-as-judge, red-teaming, model registries, drift detection and observability. 14 concepts · 144 cards
- 01 Arena Elo and Preference Ranking How pairwise human votes become a leaderboard through the Bradley-Terry model, what the Elo framing gets wrong, and the selection effects that distort arena rankings.
- 02 Custom Evals and LLM-as-Judge Why every production team eventually builds its own eval set, and how to use LLM judges without being fooled by their well-documented biases.
- 03 Epistemic and Aleatoric Uncertainty in LLMs Separating "the model does not know" from "the question has several right answers", why token-level entropy conflates the two, and how semantic clustering fixes it.
- 04 Eval-Driven Development How teams turn production failures into a regression suite, why the first fifty examples matter more than the framework, and the discipline that keeps an eval set honest as the product changes.
- 05 HELM and Holistic Evaluation Why a single accuracy number is gameable, and how Stanford's HELM, BIG-bench, and lm-evaluation-harness push evaluation toward a multi-axis picture.
- 06 LLM Observability Tooling How tracing an LLM app captures the full fan-out of model and tool calls behind one user request, and how LangSmith, Langfuse, Helicone, and Phoenix differ in what they instrument.
- 07 Model Registry, Lineage, and Reproducibility The infrastructure that answers "which dataset, code, and hyperparameters produced this checkpoint?" - and why you only miss it the first time you cannot reproduce a model.
- 08 Production Monitoring and Drift Detection How to catch silent regressions in deployed LLMs by monitoring input drift, output quality, and per-user randomised experiments before users tell you something is broken.
- 09 Public Benchmarks - MMLU, GPQA, HumanEval, MATH A tour of the academic benchmarks that anchor frontier model launches, and why most of them are saturating, contaminated, or both.
- 10 Selective Prediction and Abstention Letting a system answer only when it is likely to be right, measuring the result with a risk-coverage curve rather than accuracy, and choosing the threshold from the cost of being wrong.
- 11 Agentic Benchmarks: SWE-bench and Its Descendants What changes when a benchmark task requires many steps in a real environment, why execution-based grading is the whole point, and the contamination and harness confounds that make agent scores hard to compare.
- 12 Conformal Prediction for LLMs A distribution-free procedure that converts any confidence score into prediction sets with a finite-sample coverage guarantee, and what that guarantee does and does not promise once the predictor is a language model.
- 13 Error Bars for Evals: Sampling, Clustering and Paired Tests Why a benchmark score is an estimate with a standard error, how clustering inflates it, and why paired comparison is the single highest-leverage change to an eval report.
- 14 Red-Teaming and Adversarial Evaluation Why benign benchmark scores do not predict how a deployed model behaves under attack, and the human and automated methods used to find the failures first.
Safety & Alignment Prompt injection, jailbreaks, Constitutional AI, reward hacking and mechanistic interpretability. 25 concepts · 408 cards
- 01 Activation Steering and Representation Engineering Adding a direction to a model's residual stream at inference time changes its behaviour without any weight update, which makes concepts like refusal, sycophancy and sentiment into vectors you can add, scale, or subtract.
- 02 Alignment Evaluations and Frontier-Model Risk How frontier labs and governments measure dangerous capabilities, what an eval-gated release looks like, and where the regulatory regime sits in 2026.
- 03 Guardrail Classifiers and Content Filtering Putting a separate classifier in front of and behind the model gives you a safety layer updatable in hours rather than retrained over weeks, and it buys that agility with latency, false refusals, and a second model to keep honest.
- 04 Jailbreaks and Refusal Robustness How attackers reliably bypass model refusal training, why post-hoc filters are necessary but never sufficient, and how AILuminate measures what remains.
- 05 Prompt Injection Why LLMs cannot reliably tell instructions from data, how indirect injection weaponises retrieved content, and which partial defences are worth deploying.
- 06 Training Data Memorisation and Extraction Language models reproduce fragments of their training corpus verbatim, and an adversary with only API access can pull that data back out, which turns a training-set decision into a permanent disclosure risk.
- 07 Watermarking and Content Provenance How a statistical signal is embedded in generated text without changing its quality, why SynthID-Text scaled to production, and the robustness limits every scheme shares.
- 08 AI Control: Safety Without Trusting the Model The research agenda that assumes the model may be deliberately subverting your safeguards, and designs protocols with a red team that gets to try.
- 09 Activation Patching and Causal Tracing Swap one internal activation from a corrupted run into a clean run and measure how much of the output moves; done systematically it localises where in a network a specific computation happens.
- 10 Alignment Faking A model that infers it is in training, and that complying now prevents its values from being modified later, has an instrumental reason to comply that has nothing to do with actually holding those values.
- 11 Circuit Tracing and Attribution Graphs Replacing a model's MLP layers with a cross-layer transcoder produces a differentiable stand-in whose feature-to-feature influences can be read off as a graph, turning "which features are active" into "which features caused which".
- 12 Constitutional AI and RLAIF How Anthropic replaced human harmlessness labels with a written constitution and a critique-and-revise loop, and why this makes alignment auditable.
- 13 Data Poisoning and Backdoor Attacks An adversary who controls a small slice of the training corpus can install a trigger-activated behaviour that survives ordinary evaluation, and the number of documents required turns out not to grow with model scale.
- 14 Debate as Scalable Oversight If two strong models argue for opposing answers in front of a weaker judge, the honest side should have the easier case, which would let humans supervise systems whose answers they cannot verify directly.
- 15 Differential Privacy for Language Models The only training-time privacy defence with a formal guarantee, achieved by clipping per-example gradients and adding calibrated noise, and the concrete capability price that guarantee costs.
- 16 Evaluation Awareness and Sandbagging A safety evaluation assumes the subject cannot tell it is being evaluated, and frontier models increasingly can; that breaks the inference from a good test score to safe deployment behaviour.
- 17 Machine Unlearning in Language Models What it means to remove knowledge from a trained model, why WMDP and TOFU measure different things, and the relearning attacks that show most unlearning is suppression.
- 18 Mechanistic Interpretability Primer How sparse autoencoders extract human-interpretable features from model activations, what circuit-level analysis buys you for safety, and where the science is still contested.
- 19 Membership Inference on Language Models Deciding whether a specific document was in a model's training set is the canonical privacy attack, and on large language models it works far worse than the classical literature predicts, for reasons that are themselves informative.
- 20 Model Extraction and Weight Stealing A black-box API leaks more about the model behind it than intended, and with the right queries an attacker can recover architectural secrets or a functional clone for a tiny fraction of the training cost.
- 21 Model Organisms of Misalignment and Sleeper Agents Why safety researchers deliberately build misaligned models, what the sleeper-agent experiments showed about the durability of backdoors, and why adversarial training made things worse.
- 22 SAE Limitations and Crosscoders Sparse autoencoders were adopted on the premise that they recover a model's atomic, complete feature vocabulary, and three results published in 2024 and 2025 show they recover neither.
- 23 Scalable Oversight and Weak-to-Strong Generalisation How you supervise a model on tasks you cannot evaluate yourself, why weak labels still elicit strong capabilities, and where the analogy to superhuman supervision leaks.
- 24 Sparse Autoencoders for Feature Extraction Dictionary learning applied to activations, pulling a small set of interpretable, steerable features out of neurons that individually mean nothing.
- 25 Sycophancy, Deception, and Reward Hacking Why preference-trained models learn to please rather than to be right, what alignment faking is, and why evaluating during training can mislead you.