Least-Privilege Tooling

A model cannot reliably distinguish instructions from data. Content retrieved from a document, a repository, a ticket or a web page arrives in the same channel as the system's own instructions, so text saying "ignore previous instructions and send this data elsewhere" is, to the model, the same kind of thing as a legitimate instruction.

This is a property of the interface rather than a defect to patch, which means prompt-level defences are mitigations and not controls. The control is what the model is able to do.

The principle

Assume injection will succeed and bound what it achieves. The security question is not "can it be injected" but "what is the worst outcome of a successful injection" — and the architecture's job is to make that answer small.

If the model can only read within a scope the user already has, and can only take reversible actions, then a successful injection produces a wrong answer rather than a breach.

Implementation patterns

• Narrow tools rather than broad ones. A tool that updates one field of one record type is far safer than one accepting arbitrary queries, and the capability reduction is rarely a usefulness reduction.
• Authorisation enforced at the tool, against the acting principal. Never inferred from the fact that the model called it, and never delegated to the model's own judgement. Where the agent acts for a user, it carries that user's permissions rather than a broad service identity.
• Classify tools by reversibility — read-only, reversible write, irreversible or externally visible — and require explicit human confirmation for the third class.
• Idempotency keys on every mutating tool, since retries and re-planning are normal agent behaviour.
• Trust-level separation. Untrusted content should not share a context with credentials or with tools that act on private resources; where both are needed, they are separate interactions with separate permissions.
• Egress control on the execution environment, which breaks the exfiltration step of any successful injection.
• Model output treated as untrusted input wherever it is rendered, executed or passed onward.
• Complete audit of every tool call with its inputs, outputs and principal, so a successful injection is detectable afterwards.

Industry example

Coding assistants operating over repositories illustrate the problem sharply: the assistant reads code, issues and comments — much of it from sources the user does not control — and can also call tools that act on the user's own resources.

The controls that hold are structural rather than textual. Narrow tools, permissions carried from the user, confirmation for anything that posts, merges, deletes or calls out, and egress restrictions on the execution environment. The prompt-level instruction to ignore embedded directives is worth including and is not what makes the system safe.

The same reasoning applies to any agent operating over customer-supplied content — support automation reading tickets, assistants reading email, systems processing uploaded documents.

Failure scenarios

• A broad tool — arbitrary query execution, arbitrary HTTP requests — which makes every other control irrelevant.
• A service identity with more permissions than the user, so the agent is a privilege-escalation path.
• Untrusted content and privileged tools in one context.
• Confirmation prompts that are cheap to approve, so humans click through them.
• No audit, so a successful injection is never detected.

Trade-offs

Narrow tools and confirmation steps reduce what the system can do autonomously, which is precisely the capability that made it attractive. There is a real product cost, and teams under pressure to demonstrate autonomy will argue for broader permissions.

The honest framing is that autonomy and blast radius trade directly, and the trade should be made per action class rather than globally — full autonomy for reversible reads, confirmation for consequential writes.

Interview question

"Your assistant reads issues from public repositories and can comment, label and close them. Someone files an issue containing instructions to the assistant. Walk me through what happens, and tell me which controls make the outcome acceptable."