advanced 2 min answer

You are asked to give an internal AI agent access to the customer database, the ticketing system and outbound email so it can resolve support tickets. What is your response?

agentssecurityprompt-injectionhitl
Show the full answer Hide the answer

What the interviewer is testing

Whether you recognise a specific and well-documented security pattern, and whether you can propose a workable design instead of simply objecting.

The problem, stated precisely

The proposal combines three capabilities that are individually reasonable and jointly dangerous:

  1. Access to private data — the customer database.
  2. Exposure to untrusted content — ticket text, written by anyone on the internet.
  3. An outbound channel — email.

Any two of these are manageable. All three create an exfiltration path, because an attacker can put instructions in a support ticket, the agent reads them as instructions rather than as data, and the agent has both the data and the means to send it. Prompt injection has no complete defence at the model level — there is no reliable separation between instruction and data in a context window — so this must be handled architecturally.

The design I would propose

Break the combination. The most effective control is to remove one of the three legs:

  • The agent that reads tickets gets no direct database access. It can call narrow, parameterised tools — get_order_status(order_id) — that return only fields relevant to that ticket, scoped to the customer who raised it. It cannot issue queries.
  • The agent drafts email; it does not send it. A human approves, or a separate non-agentic process sends to a verified address on file, never to an address that appeared in the ticket.

Least privilege on every tool. Each tool has its own authorisation, scoped to the ticket's customer, read-only unless there is a specific reason otherwise. If a tool can be called with an arbitrary customer ID, an injected instruction will call it with an arbitrary customer ID.

Human in the loop by consequence. Reading and drafting need no gate. Sending an external communication, issuing a refund, or changing account data do. Present the evidence with the approval so the reviewer can actually judge, and batch carefully — approval fatigue turns the control into a rubber stamp.

Output constraints. The agent emits structured actions from a fixed schema, not free-form commands. An action outside the schema is rejected rather than interpreted.

Full audit. Every tool call with arguments and results, every retrieved document, every draft, retained and queryable. Without this, an incident cannot be scoped.

Evaluation including adversarial cases. An eval suite containing injection attempts, run in CI, so a prompt change that weakens the defences fails the build.

What I would say to the requester

Not "no". The version that works: an agent that triages, retrieves relevant history through narrow tools, drafts a reply, and hands it to an agent for approval — delivering most of the efficiency with a bounded blast radius. Then measure, and expand autonomy for the specific ticket categories where the evidence supports it.

What a strong answer adds

Noting that the same analysis applies to any agent, and that the reusable rule is: private data access + untrusted input + an outbound channel is a design to be broken up, not hardened.