Controlling Reasoning Length

The first generation of reasoning models decided for themselves how long to think, and their decisions were bad in both directions. Ask an o1-class model a trivial arithmetic question and it may spend thousands of tokens revisiting an answer it produced correctly in the first fifty; a study of this pattern found simple problems consuming large multiples of the necessary compute for no accuracy gain, and named the failure overthinking (Chen et al., 2024, Do NOT Think That Much for 2+3=?, arXiv:2412.21187). Cut thinking short and hard problems fail instead. Test-time compute is a dial that trades tokens for accuracy (see test-time compute scaling); length control is the engineering question of who holds the dial and how precisely it turns.

Budget forcing: the embarrassingly simple baseline

The s1 project demonstrated the crudest possible mechanism and made it famous. Fine-tune a 32B model on just 1,000 curated reasoning traces, then control thinking length purely at decode time: to cap thinking, force-emit the end-of-thinking delimiter when the budget is spent, obliging the model to answer with whatever reasoning it has; to extend thinking, suppress the delimiter and append the token "Wait", which reliably sends the model back to re-examine its work, often correcting itself (Muennighoff et al., 2025, s1: Simple test-time scaling, arXiv:2501.19393). Accuracy on competition math scales with the enforced budget, which made s1 the cleanest public replication of the o1-style compute-accuracy curve, with the entire mechanism visible: no reward model, no search, one magic word.

The limits are equally visible. Truncation is a cliff, not a dial: a proof cut mid-step loses the step. Extension by repeated "Wait" saturates quickly, with the model circling rather than deepening. And the budget is imposed from outside with no participation from the model, which knows best whether it is nearly done.

Learned length control

The RL generation makes length part of the objective instead of a decode-time intervention. Length Controlled Policy Optimization trains the model against prompts that state a target ("think for about 2,000 tokens"), rewarding correctness and proximity to the target jointly; the resulting L1 models interpolate smoothly across budgets, trading accuracy against cost along a curve the operator picks per query, and outperform s1-style forcing at matched budgets (Aggarwal and Welleck, 2025, L1: Controlling How Long A Reasoning Model Thinks, arXiv:2503.04697). Production APIs expose the blunter cousin: discrete effort levels or explicit thinking-token budgets, with the model trained to allocate within whatever it is granted (see hybrid reasoning models). Under the hood, penalising length in the RL reward also attacks overthinking directly, though it needs care: length penalties applied naively teach models to compress visibly while reasoning sloppily, buying shorter transcripts with wrong answers.

Choosing budgets like an operator

Length control converts a model property into a resource-allocation policy, and the allocation question has structure. Accuracy versus budget is concave: the first thousand thinking tokens buy far more than the tenth. Difficulty varies per query, so a fixed global budget overspends on easy queries and starves hard ones; the win from routing budgets by predicted difficulty is the same shape as the win from inference scaling laws generally. And thinking tokens are billed output tokens with TPOT latency, so the budget decision is simultaneously a cost line and a user-experience line: an assistant that thinks for ninety seconds has failed an interactive user even if the answer is excellent.

When it breaks

• Forced truncation fails jaggedly. Just-too-small budgets can perform worse than much smaller ones, because the model commits to an elaborate approach it cannot finish; the compute-accuracy curve is not monotone near the cliff.
• Length is a proxy, and proxies get gamed. Reward longer thinking and models pad; reward shorter and they skip verification. Both gaps between transcript and computation feed the faithfulness problem.
• Budgets leak across turns. In agent loops, per-step thinking budgets compound into episode costs, and a uniform per-step cap is wrong at both ends: exploration steps deserve thought, mechanical steps deserve none.
• Self-reported completion is unreliable. Letting the model end thinking when ready reintroduces overthinking on easy inputs and premature confidence on hard ones; the failure length control exists to fix.