DeepSeek-R1 and the Open Reasoning Recipe

When OpenAI shipped o1 in September 2024, the working assumption across the field was that frontier reasoning was a moat - the combination of data, RL infrastructure, and tacit know-how would keep it inside a handful of labs for a generation. In January 2025 DeepSeek released R1: an open-weight model with reasoning capability in the o1 class, a published paper describing the pipeline in workable detail, and six distilled variants between 1.5B and 70B parameters. The moat assumption did not survive the quarter.

R1-Zero: RL from a base model, no SFT

The most surprising result in the paper is R1-Zero: start from DeepSeek-V3 base (a 671B-total / 37B-active MoE), skip supervised fine-tuning entirely, run reinforcement learning with verifiable rewards on maths and code. Reasoning behaviours - long chains of thought, reflection ("wait, let me reconsider"), backtracking, self-verification - emerge from the RL signal alone.

Mechanics:

• Algorithm. GRPO (Group Relative Policy Optimization), a critic-free policy-gradient variant DeepSeek had introduced earlier. Removes the value-model overhead of PPO while keeping its stability.
• Reward. Programmatic - accuracy reward on verifiable answers (maths with known answers, code with unit tests), plus a format reward that the model emits its reasoning between <think> tags. No PRM, no human preference model.
• No SFT. The base model is RL-tuned directly. The "aha moment" graph in the paper shows the average response length climbing dramatically over training - the model is teaching itself to think longer.

R1-Zero hit AIME 2024 pass@1 around 71% (and ~86% with majority voting at 64), competitive with o1-preview, starting from a base model with no reasoning-specific SFT. The failure modes: poor readability (mixed languages, no structure), occasional rambling. Useful for proving the recipe, not for shipping.

R1: cold-start SFT + multi-stage RL

The shipped R1 model adds a small SFT cold-start to fix R1-Zero's readability problems, then runs further RL:

1. Cold-start SFT. A few thousand high-quality long-CoT examples to teach the model a clean reasoning format and reduce the language mixing.
2. RL with verifiable rewards for reasoning, plus a language-consistency reward to discourage mid-chain language switches.
3. Rejection sampling + SFT on the RL-tuned model's own outputs to produce a broader SFT dataset (~800k examples covering reasoning and general tasks).
4. A second RL pass for helpfulness and harmlessness on general prompts.

The result is a model with R1-Zero's reasoning capability and the conversational behaviour expected of a deployed assistant. Reported numbers (from the paper):

Within noise of o1, on most reasoning evals. Open weights. MIT licence on the distilled variants.

Distillation to smaller models

The third contribution: DeepSeek released six distilled models by SFT-tuning open base models on outputs from R1.

• DeepSeek-R1-Distill-Qwen-1.5B, 7B, 14B, 32B
• DeepSeek-R1-Distill-Llama-8B, 70B

R1-Distill-Qwen-32B outscored o1-mini on most reasoning benchmarks. R1-Distill-Llama-70B outscored Claude 3.5 Sonnet and GPT-4o on AIME and MATH-500. A 32B model with a long thinking budget, distilled from a teacher, became a credible reasoning model that runs on a single H100.

This is the line that broke the moat assumption. Distillation works on reasoning the same way it works on regular capabilities - cheaply, robustly, and at sizes hobbyists can run. Within weeks of release, derivative fine-tunes were on Hugging Face, and serving providers had R1 on their platforms.

What the paper actually claims vs what got rumoured

Reading the paper carefully against the public conversation is useful because both directions of error happened.

Actually claimed in the paper:

• Pure RL from a base model can produce strong reasoning, evidenced by R1-Zero.
• The R1 pipeline (cold-start + RL + rejection-sampling + RL) matches o1 on most public reasoning benchmarks.
• Distilling from R1 produces strong small models, beating prior open reasoners.
• Total training cost figures relate to the final RL phase, not the cost of the underlying V3 base model.

Got rumoured but not claimed:

• "R1 cost $6M to train." This conflates R1's incremental cost with the much larger DeepSeek-V3 base-model training run, which the V3 paper separately documented.
• "R1 proves you do not need lots of compute for frontier capability." It proves you do not need new lots of compute if you already have a strong base model (V3 was a serious training run).
• "Open-source has caught up to OpenAI." On reasoning benchmarks, yes, on this generation. On the next generation (o3 ARC results, agentic capability, multimodality), the gap reopened.

The honest summary: R1 is a real frontier-class reasoning model, an excellent recipe to build on, and not a refutation of the importance of base-model scale.

Why R1 broke the moat assumption

Three reasons specifically:

1. The recipe is simple enough to copy. No exotic algorithm, no proprietary data labelling, no human-in-the-loop bottleneck on PRM labels. RLVR + GRPO + a good base model is reproducible by any well-resourced lab.
2. The weights are open. Derivative work could start immediately. Within months, Qwen, Mistral, and others shipped reasoning variants citing the R1 recipe.
3. The distilled small models are deployable. R1-Distill-Qwen-32B on a single GPU put credible reasoning inside reach of anyone with $20k of hardware.

Combined: the question changed from "can a non-frontier-lab build a reasoning model?" to "how much further can the frontier labs push before the open recipe catches up again?". That dynamic now drives the field.

Where it falls down

• General-purpose helpfulness gap. R1 is strong on reasoning, weaker than the best closed models on conversational tasks, instruction following, tool use, and long-context retrieval. The reasoning-first training pipeline does not fully transfer.
• Inference cost. Long CoT means many tokens per query. The "open and cheap" framing applies to training, not necessarily to serving.
• No multimodality. R1 is text-only. Closed competitors increasingly bundle vision and audio.
• GRPO is not magic. It works well with verifiable rewards on maths and code. For domains without programmatic verifiers, the open recipe has the same labelling problem as everyone else.

Further reading

• DeepSeek-AI - DeepSeek-R1 paper (2025)
• DeepSeek-R1 GitHub repository
• Ai2 - Tülu 3 (parallel open RLVR recipe, 2024)