Reasoning Distillation and Budget Forcing

Building a reasoning model was supposed to require large-scale reinforcement learning against verifiable rewards, thousands of GPU-hours, and a reward-hacking-resistant training loop. Then a Stanford and Allen Institute team fine-tuned Qwen2.5-32B-Instruct on 1,000 examples and beat o1-preview on competition maths (Muennighoff et al., 2025, s1: Simple test-time scaling, arXiv:2501.19393).

Two ingredients, both simple, and each one tells you something different.

Ingredient one: the data is the method

The s1K dataset was distilled by filtering 59,000 candidate questions down to 1,000 on three criteria, each validated by ablation:

• Difficulty. Discard anything a small model already solves. Easy problems teach nothing about reasoning and dominate the gradient if left in.
• Diversity. Sample across mathematical subject areas rather than concentrating in whichever domain has the most available problems.
• Quality. Discard traces with formatting errors, broken reasoning, or wrong final answers.

The ablations are the interesting part. Random selection from the same 59,000-item pool, and selection on any single criterion alone, both perform substantially worse than the three-criterion filter, and training on all 59,000 examples does not beat training on the curated 1,000. The signal is in the selection, not the volume.

Ingredient two: budget forcing

At inference, control the amount of thinking directly by intervening on the decode.

To cap thinking: when the token budget is exhausted, append the end-of-thinking delimiter and force the model to answer with whatever it has. To extend thinking: when the model tries to emit the end-of-thinking delimiter, suppress it and append the string Wait instead, which makes the model continue and frequently double-check itself. Appending Wait twice extends further.

The mechanism is worth noticing. The model was never trained to respond to Wait; it is a token that, in its training distribution, precedes reconsideration. Budget forcing exploits a learned association rather than a designed control interface, which is why it works at all and also why it is brittle.

s1-32B shows a positive scaling slope on AIME24 as the forced thinking budget grows, up to a point. That makes it test-time scalable in the sense the field means: performance is a controllable function of inference compute, not a fixed property of the checkpoint. See test-time compute scaling and controlling reasoning length.

What this implies about reasoning training

The uncomfortable reading is that large-scale RL may be teaching format and policy rather than capability. If 1,000 examples of supervised fine-tuning recover most of the benefit, the base model already contained the reasoning ability, and post-training is largely eliciting a behaviour: think at length, check the work, use a particular structure.

That reading is supported by the DeepSeek-R1 distillation results, where R1's reasoning traces distilled into small dense models produced large gains without any RL on the student (DeepSeek-AI, 2025, arXiv:2501.12948). It is contradicted, or at least complicated, by the fact that the teacher required RL, and by results showing distilled students inherit the teacher's ceiling rather than exceeding it.

Both can be true. RL discovers the behaviour; supervised distillation copies it cheaply; the discovery step is expensive and happens once per frontier lab.

When it breaks

Budget forcing saturates and then reverses. Appending Wait indefinitely does not improve accuracy indefinitely. Past a point the model second-guesses correct answers into wrong ones, and the accuracy curve turns over. There is no principled stopping rule, only a tuned budget.

Small-sample distillation is domain-narrow. The s1K recipe was validated on competition mathematics, where problems are self-contained, answers are verifiable, and reasoning traces are structurally similar. There is no evidence that 1,000 examples transfer the same way for open-ended agentic work or long-horizon coding.

Contamination risk is elevated, not reduced. A 1,000-item training set drawn from public competition problems overlaps heavily with public competition benchmarks. Small-sample results demand contamination checks that large-scale training results can partly wave away on dilution grounds. See reasoning evals and contamination.

Reported gains are on a specific base model. Qwen2.5-32B-Instruct was already strong at maths. The claim is about what fine-tuning adds to that base, not about a general recipe for turning any model into a reasoner.