Reasoning Models
Test-time compute, process reward models, the o-series, DeepSeek-R1 and contamination.
15concepts
196flashcards
122minutes of reading
- 01 Chain of Thought, Self-consistency, and Search at Inference A tour of the inference-time reasoning toolkit - from zero-shot CoT prompts to MCTS-decoded reasoning trees, and when each pays for itself.
- 02 Hybrid Reasoning Models One set of weights that can answer instantly or think at length, how mode switching and thinking budgets are trained in, and why the industry converged on hybrids over separate reasoning models.
- 03 OpenAI o1, o3 and the Reasoning-Model Family What is publicly known and what is speculated about OpenAI's reasoning line, why the chain of thought is hidden, and what o3's ARC-AGI result actually proved.
- 04 Reasoning Evals and the Contamination Problem A guided tour of the reasoning benchmark canon, why each saturated faster than the field expected, and the move to live held-out evals as the contamination crisis bites.
- 05 Test-time Compute Scaling Why "thinking longer" at inference can substitute for "training bigger", how the trade-off is operationalised as a token budget, and where the strategy stops paying.
- 06 Chain-of-Thought Faithfulness Whether a model's stated reasoning is the reasoning that produced its answer, how the hint-injection test measures it, and why unfaithful CoT undermines monitoring more than it undermines accuracy.
- 07 Controlling Reasoning Length The mechanisms that set how long a model thinks, from s1's crude budget forcing to length-targeted RL, and why overthinking makes length control a quality problem rather than just a cost problem.
- 08 DeepSeek-R1 and the Open Reasoning Recipe How DeepSeek's R1 pipeline produced o1-class reasoning with an open paper, an open model, and a recipe other labs could replicate within weeks.
- 09 Deliberative Alignment Teaching a model the text of its own safety policy and training it to reason over that text before answering, which improves jailbreak robustness and overrefusal at the same time.
- 10 Inference-Time Scaling Laws and Thinking Budgets How accuracy trades against tokens spent at inference, why the optimal strategy depends on question difficulty, and what a thinking budget actually buys.
- 11 Process Reward Models and Verifiable Rewards Why scoring every step of a reasoning trace beats scoring only the final answer, and how Ai2 and DeepSeek replaced PRMs entirely with programmatic correctness checks.
- 12 Reasoning Distillation and Budget Forcing A thousand carefully chosen reasoning traces and a decoding trick can move a 32B model past o1-preview on competition maths, which says something uncomfortable about what reasoning training is actually teaching.
- 13 The Limits of Self-Correction Why asking a model to review its own answer often makes it worse, what separates intrinsic from extrinsic correction, and the oracle-label leak that inflated the early results.
- 14 Tool-Integrated Reasoning Interleaving code execution with natural-language reasoning removes an entire class of error the model cannot fix by thinking harder, and creates a new class of error it cannot see.
- 15 Verifier-Guided Search and Best-of-N Sampling many candidate solutions is only useful if you can pick the right one, and the gap between what sampling can reach and what selection can find is the central constraint on test-time scaling.