A product embeds model calls in several features and spend is growing unpredictably. What controls actually bound it?

The controls, by effect

• Attribution per feature and per customer, which is the precondition for everything else. Spend that cannot be attributed cannot be controlled, and it is usually the missing piece.
• A cascade: a small cheap model handles the majority of cases and escalates only the uncertain ones. Frequently a large reduction with no quality loss, because the expensive path runs on a fraction of traffic.
• Caching, both exact-match for repeated prompts and semantic for similar ones. For features with repetitive inputs this is a direct and large saving.
• Context discipline. Cost scales with input tokens, and prompts accumulate context that nobody has measured the value of. Trimming retrieved context and prompt boilerplate is frequently the single largest reduction and it often improves quality too, since irrelevant context degrades output.
• Output limits, since generation is typically the more expensive direction and unbounded output is both a cost and a latency risk.
• Per-feature budgets with alerts before the limit, and a defined degradation when exhausted.

The structural decision

Whether the feature needs a model at all. A meaningful proportion of model calls in production products could be a lookup, a rule, a search, or a small classifier — and they were implemented with a large model because it was faster to build.

Revisiting that after the feature proves valuable is a legitimate optimisation, and it is the one with the largest possible saving.

The measurement to run

Cost per successful outcome, not cost per call. A cheaper model that requires three attempts is not cheaper, and a cascade that escalates 40% of the time is not delivering what its design assumed.

Total spend rises with growth and tells you nothing; unit cost falling while usage grows is the only evidence the work is compounding.

The trap specific to this domain

Cost is invisible at development time. An engineer testing a prompt sees latency and quality, not the cost at production volume. Surfacing an estimated cost per call in the development loop changes prompt design more effectively than any retrospective review.