Sandboxing and Least Privilege for Agents

The most useful framing of agent security is a negative result: no amount of prompting reliably makes a model distinguish instructions from data. Every defence that lives inside the model's context is defeated by text inside that same context. AgentDojo made this measurable, running 97 realistic tasks against 629 security test cases and finding that no defence eliminated attack success while preserving utility (Debenedetti et al., NeurIPS 2024 Datasets, arXiv:2406.13352).

If the model cannot be trusted to refuse, the runtime has to be built so that a compromised model cannot do damage.

The three ingredients of a real breach

Simon Willison's framing, the "lethal trifecta", is the most economical way to hold this: an agent is exploitable when it has access to private data, exposure to untrusted content, and the ability to externally communicate, all at once. Remove any one and data exfiltration becomes hard.

That decomposition is directly actionable. A summariser that reads untrusted email and holds private data but can only write to a local file is not exfiltratable. A research agent that browses the web and posts to Slack, but has no credential, leaks nothing worth having. Architecture, not prompting, is what removes an edge.

Capability scoping

Least privilege for agents means the token the tool executes with, not the instruction in the system prompt.

• Per-tool credentials. The calendar tool holds a calendar-scoped OAuth token; it cannot read mail even if instructed to.
• Per-session narrowing. Scope down at the start of a task to only what that task's plan declared it needs, then discard.
• Read/write asymmetry. Reads are cheap to grant and hard to abuse without an egress path. Writes and sends are the ones that need gates.
• Egress allowlists. The most valuable single control. If the sandbox can only reach three hostnames, a hundred injection variants all fail at the same place.

Isolating execution

Code-executing agents need an actual sandbox, and the choice is a latency-versus-isolation trade:

Whatever the layer, the properties that matter are the same: no network by default, no host filesystem mount, a wall-clock and memory cap, and destruction after the session so state cannot carry across tenants.

Control flow that injection cannot rewrite

CaMeL is the most rigorous published answer to "what would actually work". It runs a privileged LLM that sees only the trusted user request and emits a program, and a quarantined LLM that processes untrusted content but cannot call tools. A custom interpreter enforces a capability policy over the data flowing between them, so untrusted text can influence values but never the control flow (Debenedetti et al., 2025, Defeating Prompt Injections by Design, arXiv:2503.18813). On the AgentDojo suite it solved a large majority of tasks with provable security against the injection class, at the cost of a much more constrained programming model.

The lesson generalises beyond the specific system: security comes from separating the channel that decides what to do from the channel that carries what was read.

When it breaks

Human-in-the-loop degrades into rubber-stamping. A confirmation dialog on every step trains the operator to click yes. Gates work when they are rare and legible: gate on irreversibility and on money, not on step count.

Defence-in-depth is not additive. Layering a classifier, a system prompt instruction, and a filter feels like three defences. Against an adaptive attacker they often fail correlated, because they all read the same attacker-controlled text.

The sandbox does not cover the tool. Isolating the interpreter does nothing if the agent's database tool runs as an admin role. Sandboxing is about code execution; capability scoping is about everything else, and teams routinely do the first and forget the second.