A commerce platform exposes tools for a model to call. How should the tool interface be designed, and what differs from designing an API for developers?
Show the full answer Hide the answer
What differs from a developer API
The caller cannot read documentation, hold context between calls reliably, or reason about undocumented behaviour. Everything the caller needs must be in the tool definition and in the responses.
Three consequences:
1. Names and descriptions are the interface. A developer infers meaning from documentation and examples; a model infers it from the description. Ambiguous names and vague descriptions produce wrong tool selection, which is the most common failure and is fixed in the definition rather than in the model.
2. Errors must be instructive rather than diagnostic. A developer reads an error and changes their code. A
model reads an error and retries within the same interaction. So errors should say what to do —
"order_id must be the numeric identifier, not the display reference; call find_order first" — rather than
"invalid parameter".
3. Fewer, narrower tools beat more, broader ones. A large set of overlapping tools produces wrong selection. A tool that does one thing with a small number of required parameters is chosen correctly far more often.
The safety design
Narrow permissions per tool. A tool that updates one field of one record type is far safer than one accepting arbitrary queries. The capability reduction is rarely a usefulness reduction.
Classify by reversibility — read-only, reversible write, irreversible or externally visible — and require confirmation for the third class.
Idempotency keys on every mutating tool, because retries and re-planning are normal behaviour rather than exceptions.
Authorisation enforced at the tool against the acting principal, never assumed from the fact that the model called it.
Bounded results. A tool returning thousands of rows consumes the context window and degrades everything after it. Paginate, and say in the response that more exist.
The properties that improve reliability most
- Required parameters minimal, with sensible defaults, since each required parameter is an opportunity to be wrong.
- Enumerated values rather than free text wherever the domain permits.
- Responses that state what happened, not just data — "reserved 2 units of SKU-123; 4 remain" gives the model what it needs for the next step.
- A validation-only mode for consequential tools, so a plan can be checked before execution.
The evaluation requirement
Tool selection and parameter accuracy must be measured on a held-out set of realistic tasks. Without it, a change to a tool description is an unmeasured change to system behaviour — and tool descriptions are edited constantly.