Human-in-the-Loop Authorisation Design

"Require human approval for high-impact actions" is the sixth item on OWASP's mitigation list for excessive agency, and it is the one most likely to be implemented badly (OWASP, 2025, LLM06:2025 Excessive Agency). The naive implementation shows a dialog before every tool call. After a day, the operator has approved several hundred of them, has stopped reading the arguments, and clicks through the one that matters at the same speed as the rest. The control is still in the architecture diagram. It stopped being a control some time on the first afternoon.

Approval is an attention budget, and attention is the scarcest resource in the system. A design that spends it evenly across every action has none left for the actions that deserve it. The design problem is not whether to ask a human, it is how to spend a fixed number of genuine decisions per day on the highest-consequence branches.

Classify by reversibility, not by risk score

The most useful axis is how hard the action is to undo.

Reversible and cheap: reading a document, drafting text, running a query against a replica. No approval. Log it.

Reversible with effort: creating a branch, writing a draft record, posting to an internal channel. No approval, but the action must be attributable and undoable, and someone must be told it happened.

Irreversible or externally visible: sending email outside the organisation, moving money, deleting data, merging to a default branch, granting access. These are where the budget is spent.

The distinction matters more than an abstract severity rating because it maps onto what a human can actually fix. An approval gate on an action the operator can trivially undo buys almost nothing; an approval gate on the one outbound email buys the whole incident.

Approve the plan, not the keystroke

Per-action prompts also ask the wrong question. A human shown POST /v1/transfers {"amount": 4200} has no context to judge it. A human shown "I will refund order 8812 for £42.00 because the customer reported a duplicate charge; this transfers £42.00 from the merchant account" can.

Two patterns preserve context:

Plan approval with a spending bound. The agent proposes a plan, the human approves it once, and the plan carries an envelope: which resources, which operations, what limits. Anything inside it proceeds; anything outside stops. This is the structure AP2 formalises for payments, where a user signs an intent mandate specifying price limits, timing and other conditions before the agent acts unattended (Google Cloud, 2025, Announcing Agent Payments Protocol (AP2)).

Batched review with a delay. Irreversible-but-not-urgent actions queue for a short window, are presented together with their justifications, and execute unless cancelled. One review of twelve outbound emails gets more scrutiny than twelve separate dialogs, and the window is also the window in which anomaly detection can fire.

Budgets as a second control

Approval covers the actions you anticipated. Budgets cover the ones you did not, and they degrade gracefully where approval degrades sharply.

Give each session a quota: a number of writes, a monetary limit, a count of distinct records touched, a volume of egress. Exhausting the quota escalates rather than fails silently. A support agent that has refunded twelve orders in ten minutes is either handling an outage or being driven by an injection, and in both cases stopping to ask is correct. OWASP lists rate limiting on sensitive operations in the same spirit, as damage limitation rather than prevention.

Quotas have a property approval lacks: they are enforced by counters, not by judgement, so they do not decay with operator fatigue.

When it breaks

The human is asked to validate something they cannot. Approving a SQL statement, a diff of 400 lines, or a JSON payload with an opaque identifier is not review. If the interface cannot render the consequence in the operator's own terms, the gate produces a signature and no information.

The approval is itself model-mediated. When the summary the human reads is written by the same agent whose action they are approving, an injection can write both the action and its justification. The rendering of a high-impact action should come from the enforcing system, not the model: the payment service states the amount and payee, not the agent.

Urgency is used as a lever. "The customer is waiting" is a legitimate operational pressure and an effective social-engineering payload. Time-boxed queues help because the delay is enforced rather than negotiated.

Nobody owns the queue. An approval mechanism with no staffing model becomes either a bottleneck that teams route around or an auto-approve rule added at 3 a.m. during an incident. The control's real strength is the one that survives the worst night of the quarter.