Only the tokens are recorded

At 18:04 UTC on 6 October, the official OpenAI Python client had no concept of a decision. Three and a half hours later, version 3.26.0 went up on the package index with a new resource, a handful of generated type files, and one new verb: ask a model a named question about a piece of text and get back a typed answer you can put in an if. The changelog calls it "add standalone Decisions support". The Go client shipped the same endpoint the same day. Two days later the Envoy AI Gateway learned to route it, and the pull request that did so says the endpoint is in beta and currently served for one model — a claim I could not check directly, because the provider's own developer documentation is not reachable from where this was written. Everything else here comes from the clients and the gateway, which is to say from the code that will actually make the calls.

What the endpoint does is narrow on purpose. You POST /decisions with an input — a string, or user messages carrying text and inline images — and a list of questions. A question is one of three kinds. A predicate asks how likely a statement about the input is to be true, and comes back with a probability. A choice offers between two and 255 options and comes back with the selected value, a confidence, and a probability for each option. A score supplies ordered levels and comes back with a number and a distribution over them. There are no tools, no files, no audio, no references to stored items. Answers arrive in the order the questions were asked. Each question may carry a name, and if it does, the answer carries it back.

This is a classifier endpoint. What makes it worth an architect's attention is not the classification — models have been doing that in prose for three years — but the shape of the return value. A probability with a name attached is not an opinion you have to parse. It is a value you can threshold, and a value you can threshold is a gate.

Now look at what comes back around it. The response object has exactly three fields, in both official clients, all three required: answers, model, usage. There is no identifier. There is no creation timestamp. There is one method on the resource — create — and no way to retrieve a decision afterwards, because there is nothing to retrieve it by. Compare this with the same vendor's general-purpose Responses object in the same package, which carries an id, a created-at, a status and a metadata bag, and which you can ask for again later.

Then read the usage block, which is the most carefully specified part of the whole feature. Input tokens, output tokens, total tokens; within the input details, cached tokens and cache-write tokens; within the output details, reasoning tokens. The organisation usage endpoint will bucket decisions spend at one minute, one hour or one day, grouped by project, by user, by API key, by model, by service tier, by whether it was a batch. You can discover what you spent on judgement last Tuesday between 14:00 and 14:01, attributed to the key that spent it. You cannot discover what the judgement was.

That asymmetry is the whole story, and it is not a bug in a beta. It is the natural result of billing being the provider's obligation and memory being nobody's.

Software architecture has known how to do this properly for a long time, and the vocabulary is old enough to be boring: a decision point that evaluates policy, an enforcement point that acts on the verdict, and a decision log that makes the pair accountable afterwards. The log is the part that matters under pressure. Nobody has ever been asked, in an incident review or a regulatory letter, to produce the enforcement code; they are asked what the decision was, when, on what input, under which version of which policy. That is why serious authorisation systems emit a decision record with an identifier, and why their answers distinguish deny from cannot say.

The Decisions API has that fourth answer type, and it is the sharpest detail in the specification. A question can come back as a refusal: the model declined to answer this one, and the other questions in the same request can still be answered normally. The refusal carries the question's name and nothing else. The Go client is explicit that the host may decline a question without disclosing its refusal score. So three-valued logic arrives at a two-valued call site with no number attached, and the caller has to invent the default.

You can watch the vendor invent it twice, in opposite directions, in the same week. The Agents SDK's decision recipes, added on 13 October, include a gate that reviews announcement text before a tool publishes it; there, a refusal or a missing answer means the content needs editorial review, and the publication does not happen. Alongside it sits a model router, where a refusal or a missing route means the application falls back to its more capable — which is to say more expensive — model. One recipe fails closed. The other fails costly. Both are correct for their job, and that is the point: the API does not determine the default, so the default is set by whoever is typing, usually once, usually in a hurry, usually in a constant called REVIEW_THRESHOLD that nobody will revisit. The recipes say so plainly, too: choose thresholds using labelled data from your own application. That is good advice, and it quietly concedes that the decision boundary is a float in your repository, applied to a distribution produced by a model you named with a string.

The ecosystem's first instinct tells you the rest. What the Envoy AI Gateway built, within two days, is passthrough with telemetry. Its recorder attaches six typed numeric attributes to the span: prompt tokens, completion tokens, total, cache hits, cache writes, reasoning tokens. The answers are recorded too — as one JSON blob in a single output.value string, switched off by one environment variable, and defaulting to on. The span's status is set to OK whenever the call returns, so a question the model declined to answer is, to the monitoring stack, a success. And the function that sanitises a decision request for debugging strips the input and the rubric — the instructions, the choice descriptions, the score labels — while keeping the model, the question names and the type discriminators. Read that choice as an architectural statement, because it is one: the policy is the part designed to disappear, and the name is the part that survives.

The strongest case against all of this is simple and deserves to be stated at full strength. This is a classifier, not an authorisation service, and nobody has ever demanded a decision log from a sentiment model. The documentation says so in as many words — a classifier is not an authorisation mechanism, enforce real permissions in trusted application code — and it says it in the same files that show you how to call it. Statelessness is a virtue here, not an omission: an endpoint that keeps no record cannot leak the evidence you sent it, and an identifier implies retention of exactly the material you may be contractually forbidden from leaving on someone else's disk. The caller owns the policy, so the caller owns the record. Adding an id would be theatre.

I agree with every clause of that, and I still think the warning loses, because shape beats documentation every time. The same release that carries the warning also ships a decisions tool runner: register candidate actions with ids and descriptions, ask one choice question, execute the chosen candidate, optionally verify that it worked. Its run result has three states — verified, unverified, refused. Its first line of documentation reads: select caller-defined actions with Decisions; do not treat a choice as permission. A library whose entire function is to turn model judgements into executed actions opens by asking you not to treat the judgement as permission. That is not hypocrisy. It is an honest description of an affordance, written by people who can see what will be built with it.

What will be built with it is gates. Not because engineers are careless, but because the endpoint fits the socket: a named question, a typed answer, a threshold, a branch. And every one of those gates will produce exactly one durable artefact, which is a line item. The spend is attributable to the minute, to the key, to the cache tier. The verdict lives in a trace attribute until the retention window closes, if tracing was on, if outputs were not hidden, if the span was sampled.

So the work this creates is not model work; it is records work, and it belongs to you. If you call this endpoint in a path that changes what a system does to someone — blocks a post, escalates a case, routes a refund, picks the cheaper model for a legal question — then you write the decision log before you write the branch: the question text, the model string you sent, the threshold you compared against, the answer, and the refusal as its own case rather than a fallthrough. None of that is hard. It is simply the part that no vendor has an incentive to do for you, and it is the first thing anyone will ask for.

Because the usage endpoint will tell you, to the minute, how much judgement you bought. Nothing will tell you what it said. We have built machine judgement that can be priced to the cached token and cannot be repeated, and in every review that follows a bad decision, the cost is the one question nobody asks.