An application passes user input into a model that can call tools. What is the threat, and where must the control live?

The threat

The model cannot reliably distinguish instructions from data. Text that arrives as data — a user message, a retrieved document, a web page, an email — can contain instructions, and the model may follow them. If the model can call tools, those instructions become actions.

This is not solvable by prompt engineering. Instructions telling the model to ignore instructions in the input are themselves text in the same channel, and every published mitigation of that kind has been bypassed.

Where the control must live

In the authorisation layer around the tools, not in the prompt.

• Every tool call is authorised against the user's own permissions, evaluated outside the model. If the user cannot delete that record, the tool call fails regardless of what the model was persuaded to attempt.
• Tools are scoped to the minimum, and destructive or irreversible operations are not exposed to a model acting on untrusted input at all.
• Confirmation for consequential actions, with the action described to the user in the application's own words rather than the model's.
• Untrusted content is marked as untrusted throughout the pipeline, and content originating from a third party is never given the same trust as the user's own instruction.
• Output is treated as untrusted too. Model output rendered into a page, executed as code, or used in a query is an injection vector in the conventional sense, and the conventional defences apply.

The retrieval-specific version

A poisoned document in the retrieval corpus is an injection that reaches every user whose query retrieves it. That makes corpus ingestion a security boundary: content from user submissions, crawled sources or partner feeds needs validation before it is indexed, and provenance needs to be visible in the retrieved context.

The architectural framing

Treat the model as an untrusted component that proposes actions, with the surrounding system authorising and executing them. That framing produces the right design decisions automatically, and it is the difference between a system where an injection is an inconvenience and one where it is a breach.

A system whose security depends on the model behaving correctly has no security boundary, and no amount of model improvement changes that.