An AI feature was modelled at £0.02 per interaction. Production shows £0.11. Where did the difference come from?

What the interviewer is testing

Whether you know the cost components that a single-call model omits.

The likely sources

Conversation history resent every turn. Cost per turn grows through a session rather than staying flat. A ten-turn conversation costs far more than ten times a single turn.

Retrieved context, usually the largest share of input tokens and frequently excluded from the estimate entirely.

System prompt and tool definitions, paid on every call. A dozen tool definitions can consume several thousand tokens before any work begins.

Guardrail and evaluation calls — input checks, output validation, LLM-as-judge scoring — each an additional inference.

Agent loops. Several model calls per user request, and many more when a loop goes wrong.

Retries after validation failure, doubling the cost of the requests that need them.

Reasoning tokens on models that produce them: billed, and invisible in the response.

The fix for measurement

Instrument per interaction, not per API call, attributing every call in the trace back to the originating user action. That instrumentation belongs in the AI gateway, and without it the number cannot be known.

The fix for cost

Model routing — send classification, extraction and simple responses to a small cheap model and reserve the large one for genuine reasoning. This is usually the largest single reduction available, and teams consistently find a smaller model matches on more traffic than they predicted.

Trim retrieval — rank and truncate rather than passing everything. Fewer, better passages improve quality and cost.

Summarise or window conversation history rather than replaying it verbatim.

Cache identical and semantically similar requests.

Bound agent loops with a hard iteration limit.

What a strong answer adds

The commercial urgency: the feature was priced or budgeted against the modelled figure, so the discrepancy determines viability at scale. Discovering it in a pilot is far better than after committing to a pricing model — and the ceiling controls are much easier to add before users grow accustomed to their absence.

Common weak answers

Negotiating provider pricing. Switching models without measuring where the tokens go.