Concept library
446 concepts across 8 domains and 36 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
07
Reasoning, Evaluation & Safety
Models that think longer, the evals that measure them, and the failure modes that matter.
3tracks
31concepts
157cards
4.3hreading
Reasoning Models Test-time compute, process reward models, the o-series, DeepSeek-R1 and contamination. 9 concepts · 46 cards
- 01 Chain of Thought, Self-consistency, and Search at Inference A tour of the inference-time reasoning toolkit - from zero-shot CoT prompts to MCTS-decoded reasoning trees, and when each pays for itself.
- 02 OpenAI o1, o3 and the Reasoning-Model Family What is publicly known and what is speculated about OpenAI's reasoning line, why the chain of thought is hidden, and what o3's ARC-AGI result actually proved.
- 03 Reasoning Evals and the Contamination Problem A guided tour of the reasoning benchmark canon, why each saturated faster than the field expected, and the move to live held-out evals as the contamination crisis bites.
- 04 Test-time Compute Scaling Why "thinking longer" at inference can substitute for "training bigger", how the trade-off is operationalised as a token budget, and where the strategy stops paying.
- 05 Chain-of-Thought Faithfulness Whether a model's stated reasoning is the reasoning that produced its answer, how the hint-injection test measures it, and why unfaithful CoT undermines monitoring more than it undermines accuracy.
- 06 DeepSeek-R1 and the Open Reasoning Recipe How DeepSeek's R1 pipeline produced o1-class reasoning with an open paper, an open model, and a recipe other labs could replicate within weeks.
- 07 Inference-Time Scaling Laws and Thinking Budgets How accuracy trades against tokens spent at inference, why the optimal strategy depends on question difficulty, and what a thinking budget actually buys.
- 08 Process Reward Models and Verifiable Rewards Why scoring every step of a reasoning trace beats scoring only the final answer, and how Ai2 and DeepSeek replaced PRMs entirely with programmatic correctness checks.
- 09 The Limits of Self-Correction Why asking a model to review its own answer often makes it worse, what separates intrinsic from extrinsic correction, and the oracle-label leak that inflated the early results.
Evaluation & MLOps Benchmarks, LLM-as-judge, red-teaming, model registries, drift detection and observability. 11 concepts · 57 cards
- 01 Arena Elo and Preference Ranking How pairwise human votes become a leaderboard through the Bradley-Terry model, what the Elo framing gets wrong, and the selection effects that distort arena rankings.
- 02 Custom Evals and LLM-as-Judge Why every production team eventually builds its own eval set, and how to use LLM judges without being fooled by their well-documented biases.
- 03 Eval-Driven Development How teams turn production failures into a regression suite, why the first fifty examples matter more than the framework, and the discipline that keeps an eval set honest as the product changes.
- 04 HELM and Holistic Evaluation Why a single accuracy number is gameable, and how Stanford's HELM, BIG-bench, and lm-evaluation-harness push evaluation toward a multi-axis picture.
- 05 LLM Observability Tooling How tracing an LLM app captures the full fan-out of model and tool calls behind one user request, and how LangSmith, Langfuse, Helicone, and Phoenix differ in what they instrument.
- 06 Model Registry, Lineage, and Reproducibility The infrastructure that answers "which dataset, code, and hyperparameters produced this checkpoint?" - and why you only miss it the first time you cannot reproduce a model.
- 07 Production Monitoring and Drift Detection How to catch silent regressions in deployed LLMs by monitoring input drift, output quality, and per-user randomised experiments before users tell you something is broken.
- 08 Public Benchmarks - MMLU, GPQA, HumanEval, MATH A tour of the academic benchmarks that anchor frontier model launches, and why most of them are saturating, contaminated, or both.
- 09 Agentic Benchmarks: SWE-bench and Its Descendants What changes when a benchmark task requires many steps in a real environment, why execution-based grading is the whole point, and the contamination and harness confounds that make agent scores hard to compare.
- 10 Error Bars for Evals: Sampling, Clustering and Paired Tests Why a benchmark score is an estimate with a standard error, how clustering inflates it, and why paired comparison is the single highest-leverage change to an eval report.
- 11 Red-Teaming and Adversarial Evaluation Why benign benchmark scores do not predict how a deployed model behaves under attack, and the human and automated methods used to find the failures first.
Safety & Alignment Prompt injection, jailbreaks, Constitutional AI, reward hacking and mechanistic interpretability. 11 concepts · 54 cards
- 01 Alignment Evaluations and Frontier-Model Risk How frontier labs and governments measure dangerous capabilities, what an eval-gated release looks like, and where the regulatory regime sits in 2026.
- 02 Jailbreaks and Refusal Robustness How attackers reliably bypass model refusal training, why post-hoc filters are necessary but never sufficient, and how AILuminate measures what remains.
- 03 Prompt Injection Why LLMs cannot reliably tell instructions from data, how indirect injection weaponises retrieved content, and which partial defences are worth deploying.
- 04 Watermarking and Content Provenance How a statistical signal is embedded in generated text without changing its quality, why SynthID-Text scaled to production, and the robustness limits every scheme shares.
- 05 AI Control: Safety Without Trusting the Model The research agenda that assumes the model may be deliberately subverting your safeguards, and designs protocols with a red team that gets to try.
- 06 Constitutional AI and RLAIF How Anthropic replaced human harmlessness labels with a written constitution and a critique-and-revise loop, and why this makes alignment auditable.
- 07 Machine Unlearning in Language Models What it means to remove knowledge from a trained model, why WMDP and TOFU measure different things, and the relearning attacks that show most unlearning is suppression.
- 08 Mechanistic Interpretability Primer How sparse autoencoders extract human-interpretable features from model activations, what circuit-level analysis buys you for safety, and where the science is still contested.
- 09 Model Organisms of Misalignment and Sleeper Agents Why safety researchers deliberately build misaligned models, what the sleeper-agent experiments showed about the durability of backdoors, and why adversarial training made things worse.
- 10 Scalable Oversight and Weak-to-Strong Generalisation How you supervise a model on tasks you cannot evaluate yourself, why weak labels still elicit strong capabilities, and where the analogy to superhuman supervision leaks.
- 11 Sycophancy, Deception, and Reward Hacking Why preference-trained models learn to please rather than to be right, what alignment faking is, and why evaluating during training can mislead you.