An agent reads customer emails and can issue refunds. What is the threat and how do you bound it?

What is being tested

Whether you treat prompt injection as a privilege problem rather than a filtering problem.

The threat

The agent processes untrusted content — an email written by anyone — and has a consequential tool — issuing refunds. An email containing instructions can attempt to redirect it.

Because models process instructions and data in the same channel, there is no reliable way to make a model ignore instructions in its input. Filtering raises the bar; it does not close the hole. Attacks can be encoded, translated, embedded in images or spread across messages, and a design relying on detection is relying on something that will eventually be bypassed.

So the correct question is not "can it be tricked" — assume it can — but "what can it do if it is?"

How to bound it

1. Least privilege on the tools. The agent should not be able to issue arbitrary refunds. Bound the amount, bound the rate, restrict to the customer whose email is being processed, and exclude accounts above a threshold.

2. Human confirmation for consequential actions. Any refund above a threshold requires approval, with the action and its basis shown in terms the human can evaluate. This is a primary control here, not a courtesy.

3. Separate trust levels. Split into a low-privilege component that reads and summarises untrusted email, emitting structured output, and a higher-privilege component that acts on validated, schema-constrained output. The component touching untrusted content never holds the dangerous tool.

4. Constrain what the model can emit. A defined schema, an enumerated set of actions, validated before anything executes. This converts "the model said something" into "the model selected one of these permitted operations".

5. Authorisation checked outside the model, on every action. The model's assertion about who the customer is carries no weight; the enforcing layer verifies independently that this email address corresponds to this account and this order.

6. Full audit and anomaly detection. Every action logged with its trigger. Alert on unusual patterns — a spike in refunds, refunds to unusual destinations.

What a strong answer adds

That the same reasoning applies to retrieval: if the agent can search internal documents, retrieval must enforce the requesting context's permissions, or an injected instruction can exfiltrate data the requester was never entitled to.