Variety is the billable part

Read. Write. Uncached. Other.

Those are the four values a working group has settled on for what a token can do with a cache, and people have been arguing about them since March. The feature request that started it opens in the register of bookkeeping: GenAI API providers price cached and uncached tokens differently, creating distinct pricing dimensions within the same token type. On 16 September a pull request settled the shape of the answer — two properties, TokenCacheAction and TokenDirection, rather than one string folding both together — and merged into the branch where the proposal is being assembled. It is not in the working draft. It is not in any released version of the standard; the changelog through v1.4, ratified in June, does not mention tokens or caches at all.

This is FOCUS, the FinOps Open Cost and Usage Specification, which exists so that a cost record from one provider can be read next to a cost record from another without a translation layer per vendor. Billing schemas are not usually interesting. This one is, because a billing schema is where an industry writes down what it thinks it is selling, and what this proposal writes down is that the industry is no longer selling tokens. It is selling tokens that have not been seen before.

Everything below comes out of the committee's own public record — the feature request, the two pull requests, the draft column text on both branches. There is no independent reporting to weigh it against, and the network available for this piece reached very little else, which matters most for the provider prices quoted further down: those are the proposal's survey of its members' price lists, not something checked against the providers' own pages.

Start with what the draft text actually requires, because the requirement level is half the story. TokenCacheAction is defined as the "interaction of the metered tokens with a cache," a string with four allowed values. TokenDirection is the "direction of the metered tokens, into or out of the model," and takes Input or Output. Both are SHOULD. The only MUST language in the draft says when they are forbidden: a SKU price that does not meter request tokens must not carry a cache action, and — this one is worth pausing on — a price that meters request and response tokens together must not carry a direction at all. Where a provider has blended the two into one meter, the standard's response is silence rather than a guess. That is the right call and it is also an admission. A vocabulary for describing someone else's internals can only ever be as fine-grained as their invoice already was.

Both properties live inside SkuPriceDetails, a JSON object of key-value pairs describing the SKU you were charged under. The column is explicit that it excludes "dynamic or negotiable pricing elements such as unit price amounts; currency; temporal validity; and contract- or negotiation-specific factors." It describes the price, not the purchase. Which means the cache status is not an observation about your request. It is a label on the meter that caught it.

Now the money. The proposal's own comparison of provider pricing records cache reads at roughly a tenth of the ordinary input rate across the providers it surveys, and cache writes priced above plain input: for Anthropic, it records 1.25x for the five-minute lifetime and 2x for the hour. Gemini it records as charging a one-time write plus a per-token-hour rent on what is held. OpenAI, it notes, did not charge for writes before GPT-5.6.

Read those numbers as an architect rather than as a buyer. A cache read is not a discount on a token; it is a different token with its own SKU, one order of magnitude cheaper. A cache write is not an optimisation; it is a priced bet that the same content will come back before the entry expires, and if it does not, you have paid a premium for nothing. The unit of AI cost stopped being the request some time ago and is now, formally, the token's relationship to something that happened earlier. A system's bill is decided less by what it asks than by how much of what it asks it has asked before.

Which gives the practical consequence, and it is not subtle once stated: the part of a request that varies is the part you pay full rate for. The stable scaffolding — the system prompt, the tool definitions, the policy text, the long boilerplate that everyone has been told to trim — is the cheap part, once it has been paid for once. The expensive part is whatever is different this time. The user's own data. The retrieved passage. The fresh context. The personalisation. Every property that makes a response worth having instead of generic lands on the full-price side of the ledger, and the discount belongs to the text your system says to everyone.

The two caching models the feature request describes make this land differently in different shops. It characterises Anthropic's as explicit opt-in and OpenAI's as automatic. Where caching is automatic, the discount happens to you, and the varying tail of your prompt quietly becomes your dominant cost without any decision being taken. Where it is opt-in, someone decides — and that someone is usually a developer choosing where to draw a line in a prompt template, which is now among the more consequential unit-economics decisions in the system and is not treated as one anywhere.

Then there is the word the committee could not agree on. Uncached is defined as applying "when the SKU Price meters tokens consumed from a request without distinguishing a cache interaction." Read it twice. It does not mean the cache missed. It means the meter did not say. Reviewers pushed for None instead, on the grounds that no cache interaction is being metered, and the question was left open. Either way, the value that will appear most often in real data is the one that carries no information about caching whatsoever — the residual bucket for every provider and every meter that has not decomposed its own pricing. Other, meanwhile, is the escape hatch for "a cache-related token charge to which none of the other allowed values apply," which is where every genuinely new caching scheme will arrive and where comparison will go to die.

The honest counterargument is strong, and it is not that any of this is wrong. It is that the price is tracking something real. A cache read skips work the provider would otherwise do; the cost difference is not a vendor's preference, it is arithmetic about compute that did or did not happen. Rewarding reuse is what caching has always done, everywhere in computing, and nobody calls a CDN coercive for charging less on a hit. Designing prompts so the stable content comes first is ordinary hygiene with real savings attached, available today, requiring no new discipline and no new theory. Writing it into an open standard so that practitioners can compute a hit rate without vendor-specific parsing is straightforwardly good, and the same committee is proposing exactly that: cache hit rate, input-to-output ratio, cache cost efficiency.

All of that is true. The difficulty is that the discount is indifferent to content. It pays the same for repetition that serves the user and repetition that flattens them, and there is no field anywhere in this schema — by design, since it excludes the dynamic and the negotiated — for what the variation bought. A column that says a token was uncached cannot say whether the thing that made it uncached was a customer's name, a retrieved document that changed the answer, or a template nobody bothered to stabilise. Once that ratio is chartable by team, it becomes a number with an owner, and numbers with owners get moved. The available move is to make requests more like each other.

That is a product decision. It is about to be made, in many organisations, by an accounting schema and a dashboard, and the people who will make it are not the people who would recognise it as a product decision. This is the recurring shape of the AI build-out: the measurement apparatus arrives years before the design vocabulary, so the first thing anyone can see becomes the first thing anyone optimises. We got a standard column for cache behaviour before we got a serious professional argument about how much sameness a system should be designed to have.

And it is still SHOULD. A provider may report none of this, and the comparability that justifies the whole exercise will arrive unevenly, later, and never for the meters that blend directions together. So the pressure to raise a cache hit rate will land on architecture well before the number is trustworthy enough to defend a design in front of anyone.

The bill, eventually, will tell you what your variety cost. Nothing in it will tell you what it was for.