Cost and Throughput Engineering on the Claude API

Most teams tune the wrong dial first. They shorten the system prompt, which saves a few hundred cached tokens billed at a tenth of list price, while the same workload runs at maximum reasoning effort on every request including the trivial ones. The levers differ by orders of magnitude in effect, and they are not interchangeable: some trade latency, some trade quality, and one trades nothing at all.

Reasoning effort is usually the biggest single lever

output_config.effort controls how much the model thinks and, on agentic work, how many tool calls it consolidates. It takes low, medium, high, xhigh, and max, and defaults to high. Lower effort produces fewer and more consolidated tool calls, less preamble, and terser confirmations; higher effort produces more thorough exploration.

The important and slightly counterintuitive part: on agentic workloads, higher effort up front can reduce total cost by cutting the number of turns needed, so the relationship between effort and spend is not monotonic. This makes effort a per-route decision to be swept against your own evaluations rather than a global setting. Routing classification and extraction at low while leaving hard multi-step work at high is where most of the realisable saving lives.

Batch processing is the free 50%

The Message Batches API processes requests asynchronously at half price, accepting up to 100,000 requests or 256 MB per batch, with most batches finishing within an hour and a hard ceiling of 24 hours. Results stay available for 29 days.

This is the only lever on the list that costs nothing in quality. Its price is latency, so it applies to work with no user waiting: nightly evaluation runs, backfills, bulk classification, document enrichment, synthetic data generation. Two operational details matter. Results arrive in any order, so key them by custom_id and never by position. And caching still applies inside a batch, so a shared document placed in a cached system prefix across thousands of requests compounds the batch discount with the cache read discount.

Model tiering, and the caching interaction

Routing simple work to a smaller, cheaper model is obvious in principle and has a non-obvious constraint in practice: caches are model-scoped, so switching models mid-conversation discards the cached prefix and pays full price to rebuild it on the new model. The pattern that works is not to switch the main loop but to keep it on one model and spawn a subagent on the cheaper model for a bounded sub-task, leaving the parent's cache intact.

Task budgets and deferred tools

A task budget (output_config.task_budget, minimum 20,000 tokens, behind a dated beta as of mid-2026) tells the model how many tokens it has for an agentic loop. The server injects a countdown the model can see, so it paces itself and finishes gracefully rather than being cut off. This is categorically different from max_tokens, which is an enforced per-response ceiling the model knows nothing about. A budget is advisory; a max_tokens cut is a truncation.

Deferred tool loading addresses a different waste: an agent with sixty tools pays for sixty schemas on every request even though three are relevant. Marking tools defer_loading: true and adding a tool-search tool lets the model load schemas on demand, and because schemas are appended rather than swapped, the prompt cache survives.

When it breaks

• You cannot tune what you do not measure. Per-route token accounting — split into uncached input, cache writes, cache reads, and output — is the prerequisite for every decision above. Without it, optimisation is guesswork with a confident narrative.
• Aggressive effort reduction is a silent quality regression. Lower effort scopes work to exactly what was asked, which is a feature on simple tasks and under-thinking on complex ones. Sweep against evals, not against the bill alone.
• Batch latency is a product decision, not just an engineering one. A 24-hour worst case has to be acceptable to the workflow; retrofitting async into a synchronous UX costs more than the tokens saved.
• Truncation is the most expensive kind of saving. A response cut off at max_tokens is usually retried at full cost, so a tight ceiling can raise spend. Give headroom and control depth with effort instead.
• Rate limits are a separate budget from money. Newer models often sit in their own rate-limit pool, so shifting traffic to one neither inherits nor frees headroom on another. Check the limits for the specific model before moving volume.