advanced 2 min answer

An agent that researches and summarises occasionally runs for 40 minutes and costs £30 for one request. How do you contain it?

agentscostcontrol
Show the full answer Hide the answer

What the interviewer is testing

Whether you treat an agent loop as an unbounded computation requiring hard limits.

Why it happens

The agent is in a loop that does not converge: repeating a search that returns nothing useful, alternating between two approaches, calling a failing tool and retrying, or pursuing a decomposition that keeps generating subtasks.

The model has no reliable sense of when to stop, and nothing in the architecture is stopping it.

The controls

A hard iteration limit, enforced by the orchestration code — not by an instruction in the prompt. Ten to fifteen steps for most tasks. On reaching it, return the best answer available with an indication that the task was truncated.

A token budget per request, enforced by the gateway, aborting when exceeded.

A wall-clock deadline, propagated so the agent knows how much time remains and can be told to conclude.

Loop detection: if the same tool is called with the same or near-identical arguments repeatedly, break. This catches the most common non-convergence pattern cheaply.

A cost ceiling per request and per user per period.

Progress evaluation: after every few steps, assess whether new information is being gathered. If not, stop.

The design change that matters more

Prefer bounded workflows to open-ended agency. Many tasks framed as agent problems are actually a known sequence with a decision point or two. A workflow with defined steps and a model at each is predictable in cost, latency and behaviour, and it is far easier to evaluate and debug.

Reserve open-ended agent loops for genuinely open-ended tasks, and treat them as expensive.

What a strong answer adds

Making the cost visible to the user for long-running tasks — showing progress and steps taken, with the ability to stop. That converts a mysterious 40-minute wait into something a user can control, and it produces the feedback that identifies which task types are pathological.

Common weak answers

Instructing the model to be efficient. Raising the timeout.