Agent Permissions and Blast Radius
How to design tool access for a component you must assume is compromised, why capability scoping beats identity-based permissions here, and the reversibility test that decides what needs confirmation.
An agent is a program whose control flow is determined by a language model that an attacker may be able to steer. Designing its permissions is therefore not an access control problem in the usual sense; it is the problem of deciding what damage is acceptable if the component is fully compromised, because you cannot rule that out.
Scope to the task, not to the user
The default arrangement gives the agent the user's permissions, on the reasoning that it acts on their behalf. That maximises blast radius: a user with broad access to email, files and internal systems hands all of it to a component that reads untrusted content.
The better arrangement scopes to the task. An agent summarising a document needs read access to that document, not to the document store. An agent booking travel needs the booking API, not the payment API. Deriving permissions from the task rather than the actor makes them narrow by construction and makes over-permission visible at design time.
Where the task is not known in advance, the useful pattern is to grant capabilities incrementally: begin read-only, escalate on explicit user action, and drop escalated permissions when the step completes.
The reversibility test
The property that should determine whether an action needs confirmation is not sensitivity but reversibility, and specifically whether the user could detect and undo the action.
Reading a file is reversible in the sense that nothing changed. Writing to a scratch directory is trivially undone. Sending an email is not reversible and is externally visible. Deleting a production record, making a payment, posting publicly and modifying access controls are all irreversible and consequential.
The design that follows is to allow reversible actions freely, require confirmation for irreversible ones, and forbid the small set that should never be automated. This concentrates the human's attention where it is worth something, which is what keeps confirmation from degrading into reflex.
Structural controls
Egress restriction is the highest-value control for an agent handling untrusted content, because most injection payloads need to send data somewhere. An allowlist of destinations turns a successful injection into a failed exfiltration.
Isolation of untrusted content. A session that has processed attacker-controlled text should not also hold credentials or high-privilege tools. Separating the parsing context from the acting context bounds what a compromise reaches.
Sandboxed execution. Code produced by a model runs in a container with no network, no credentials, a read-only filesystem except for a scratch mount, and resource limits. Treating generated code as untrusted input to an execution service, rather than as trusted code, is the correct framing.
Complete audit. Every tool call, its arguments, its result, and the model version and prompt that produced it, logged. This does not prevent anything and it is what makes an incident investigable, which is the difference between a bounded incident and an unbounded one.
When it breaks
Capabilities compose into more than their parts. Read access to internal documents and the ability to make an outbound request are individually reasonable and jointly an exfiltration path. Permission review has to consider combinations, which is harder than reviewing each grant and is where the real risk sits.
Sandboxes leak through their interfaces. A container with no network still has whatever the host passes in: environment variables, mounted paths, and the tool results the agent was given. The sandbox bounds code execution and not information flow.
Long-running agents accumulate state. An agent that persists across sessions carries whatever it was told earlier, so an injection in one session can influence a later one. Memory is an attack surface, and expiring or scoping it is part of the permission design.
Users cannot evaluate what they approve. A confirmation prompt showing an opaque tool call with a long argument does not enable a decision. Confirmations must present the consequence in terms the user understands, and one that does not is a compliance artefact rather than a control.
14 flashcards for this concept
Click a card to reveal the answer.