A platform builds an agent that plans and executes multi-step tasks against business systems. Which architectural constraints are non-negotiable?
Show the full answer Hide the answer
The non-negotiable constraints
1. The agent acts with an identity and its authorisation is enforced by the systems it calls. Not by the agent's own reasoning. An agent that decides for itself what it may do has no security model — authorisation must be checked at the tool, against the acting principal, exactly as it would be for a human user.
Where the agent acts on a user's behalf, it should carry that user's permissions, not a broad service identity.
2. Every action is bounded and reversible, or requires confirmation. Actions are classified: read-only, reversible writes, and irreversible or externally-visible effects. The third class — sending a message, issuing a refund, deleting a record, calling a partner — requires explicit human confirmation or a hard policy allowing it.
3. Idempotency on every action. Agents retry, re-plan and repeat steps. Without idempotency keys, a retry produces a duplicate refund or a duplicate message.
4. Hard limits on the loop. Maximum steps, maximum wall-clock time, maximum cost. An agent without a budget can consume unbounded resources pursuing a task it cannot complete.
5. Complete auditability. Every step, every tool call, its inputs and outputs, the acting principal, and the reasoning if available. When an agent does something unexpected, this is the only way to understand it — and for a business system it is also a compliance requirement.
6. Tool outputs treated as untrusted input. Content retrieved from a document, a web page or a ticket can contain instructions. If the agent acts on them, that is prompt injection with real consequences, and the defence must be structural rather than a plea in the prompt.
The design that follows
Narrow tools with narrow permissions, rather than broad ones. A tool that can update one field of one record type is far safer than one that can execute arbitrary queries, and the reduction in capability is usually not a reduction in usefulness.
A plan-then-confirm interaction for anything consequential: the agent proposes a sequence, the human approves, execution proceeds within the approved bounds.
The framing
An agent is an untrusted client of your systems that happens to be fast and persuasive. Every control you would apply to an external integration — authentication, authorisation, rate limits, idempotency, audit, bounded scope — applies, and the reasoning capability changes none of it.