Tool-Integrated Reasoning

A model asked to compute \(\binom{40}{17}\) inside a chain of thought will produce a plausible integer. It is usually wrong, and no amount of additional reasoning fixes it, because the model is pattern-matching a number rather than computing one. Give the same model a Python interpreter and the answer is exact, in one line, every time.

Tool-integrated reasoning is the structural response: instead of choosing between natural-language reasoning and program synthesis, interleave them within a single trajectory, so the model reasons about what to compute in language and delegates the computation itself.

The trajectory format

ToRA established the pattern that most subsequent systems follow (Gou et al., 2024, ToRA, arXiv:2309.17452, ICLR 2024). A trajectory alternates three element types:

1. Rationale in natural language: what to do next and why.
2. Program in a tool-callable language, emitted as an action.
3. Output: the interpreter's actual result, appended to the context by the harness, not generated by the model.

The third element is the load-bearing one. Because the output is inserted rather than sampled, the model conditions on ground truth for every computed quantity, and errors cannot compound the way they do in a pure chain of thought.

Training is two-stage. Imitation learning on trajectories collected from a stronger model gives the basic format. Output space shaping then improves it: sample many trajectories per problem, keep the valid ones, and use a teacher to correct invalid ones into valid ones, so the training set covers recovery behaviour and not just success. ToRA-7B reached 44.6 percent on MATH, and ToRA-Code-34B exceeded 50 percent, the first open model to do so, beating GPT-4's chain-of-thought result on that benchmark at the time.

Why the division of labour is the point

The two modes fail in complementary ways, and the gain comes from that complementarity rather than from either mode being better.

Natural-language reasoning is good at plan formation, case analysis, and deciding which theorem applies. It is unreliable at arithmetic, symbolic manipulation, and anything requiring many exact steps, because every intermediate is a sampled token that can be wrong.

Program execution is exact and can loop, but it cannot decide what to compute, and it fails opaquely when the framing is wrong. A correct program computing the wrong quantity produces a confident wrong answer with no visible error.

Interleaving lets each mode do what it is good at, and lets the model see the result of its own action before choosing the next one, which is what turns a single-shot generation into a feedback loop.

Where it has gone since

The pattern generalised beyond a calculator. Search-R1 and similar systems train models with reinforcement learning to interleave retrieval calls with reasoning, so the model learns when to look something up rather than following a fixed retrieve-then-generate pipeline (Jin et al., 2025, arXiv:2503.09516). Frontier reasoning models now use tools inside the thinking block itself rather than after it.

The important shift is from tool use as an output step to tool use as a reasoning step. When the interpreter result lands inside the chain of thought, subsequent reasoning is conditioned on it, and the model can revise its plan based on what it found. When tools are called only at the end, they can validate but not redirect.

When it breaks

Latency and cost change shape. Each tool call is a round trip that stalls decoding, and prefix caching gains are limited because each inserted output invalidates the suffix. A trajectory with eight tool calls has eight synchronisation points, which is why tool-integrated reasoning is far more expensive per token of useful output than its token count suggests.

Silent wrong-framing errors replace loud arithmetic errors. The failure mode moves rather than disappearing. A program that runs cleanly and computes the wrong integral gives a precise answer to the wrong question, and neither the model nor the harness has any signal that something went wrong.

Sandboxing is now on the critical path. Executing model-generated code with real inputs is a security boundary, and the model's code is untrusted by construction. See agent sandboxing and least privilege.

Verifiability is uneven. The benchmarks where tool-integrated reasoning shines are those with checkable answers. In domains where correctness is a judgement call, the interpreter provides no ground truth and the whole advantage evaporates, leaving only the cost. See RL from verifiable rewards and structured output coercion.