You cannot bill a histogram

The invoice arrives and it is larger than last month. Finance wants to know which product line spent it. You have, in principle, everything: your services are instrumented, the traces are flowing, there is a dashboard with token counts on it. So you go and look, and you discover that you own two numbers about the same event, that they disagree, and that neither of them is an answer.

This is not a tooling failure. It is written into the specification, and the specification is candid about it.

Since June, when the OpenTelemetry project moved its generative-AI conventions out of the main semantic-conventions repository and into one of their own, the GenAI vocabulary has been evolving on its own clock. Watch what it has been doing with cost. On 20 August a change landed adding token-usage breakdowns per modality, per cache status and per phase — text, image and audio input and output, cache reads, cache writes, reasoning tokens. Its author wrote that this "gives almost precise cost for individual operations." Fourteen usage attributes now sit on an inference client span. Metrics, the change noted, "will be addressed in #374."

Pull request 374 has been open since 12 July.

So the state of the art, today, is this. The metric — gen_ai.client.token.usage, a histogram, aggregated rather than sampled, the closest thing in the pipeline to a complete count — carries an attribute called gen_ai.token.type whose entire vocabulary is input and output. It will tell you exactly how many input tokens a given model consumed. It has no way to say how many of them were cache reads.

That distinction is not a rounding error. In the price table LiteLLM maintains and a good deal of cost tooling actually reads, a cache-read input token on GPT-5, Claude Sonnet 4.5 and Gemini 2.5 Pro costs a tenth of an ordinary input token; on Sonnet, writing to the cache costs a quarter more than not using it. Reasoning tokens are billed as output, and output on GPT-5 runs eight times input. Multiply the complete metric by a headline price and your answer is wrong by a factor that depends on cache hit rate, reasoning depth and modality mix — which is to say, wrong by exactly the amount that good engineering moves. The team that spent a quarter restructuring prompts to make the cache work gets a number indistinguishable from the team that did not.

The priceable detail exists. It is just on the other signal. And the other signal is the one the tracing specification defines as reducible: sampling, it says, is "a mechanism to control the noise and overhead introduced by OpenTelemetry by reducing the number of samples of traces collected and sent to the backend." That is not a criticism of OpenTelemetry. It is the design premise that made distributed tracing affordable at all. Traces were never a ledger. They were a way of looking closely at a representative few.

The conventions are clear-eyed about where this leads, and the clarity is worth reading directly. On 27 August the cache-usage attributes were removed from internal agent spans, on the grounds that an agent span accumulates usage across a whole call tree, and "a cache breakdown at that level mixes models with different cache semantics and pricing, so the number is not interpretable on its own." Consumers who need cost allocation, the change said, should aggregate from the inference client spans instead.

Read that as an architect rather than as a contributor. The standard's own guidance for answering "what did this agent cost" is: reconstruct it from the leaves of the trace. Which is a query over a sampled population, run against a signal whose retention is set by whoever owns the observability budget.

There is a second gap underneath the first, and it explains why the obvious fix is not available. The metric's specified attributes are the operation, the provider, the model requested, the model that answered, the server address and port, and the token type. There is no tenant, no customer, no feature, no agent. Add one and you have not extended a metric so much as multiplied it: a metrics pipeline carries one time series per attribute combination, and the same project demoted network.peer.address to opt-in on two HTTP connection metrics in the same June release, on the grounds that a value like that "causes unbounded growth of metric streams." The dimension that turns a number into a chargeback is the dimension a metrics system cannot afford to carry. Attribution migrates onto the trace not out of laziness, then, but because the trace is the only signal in the stack that tolerates identity — and the price of tolerating identity is that it may be thrown away.

Both halves of that earlier sentence matter, and the second is the one that gets missed. Sampling is not, in fact, the default: the specification's default sampler records everything. Completeness here is a deployment choice, not a property of the signal — and that is precisely the problem. Somebody decides. In most organisations the somebody is a platform engineer, some months into a project to get the tracing bill down, choosing a ratio or a tail-sampling policy that keeps the errors and the slow requests and drops the boring successful ones. That is correct behaviour for debugging. It is catastrophic for accounting, because the boring successful requests are where the money is, and nothing in the change ticket says the number finance uses will move.

And the pressure is worst where the question is hardest. An agent invocation is not one span. It is an invocation span, an inference span per model call, a span per tool execution, more of the same for every sub-agent it delegates to. Agentic workloads emit more spans per unit of work than almost anything else in a modern system, which means they are the first candidate for aggressive sampling — and they are also the workloads whose cost nobody can predict. You sample the traces because the agents are expensive. You needed the traces to find out why the agents are expensive.

There is a serious objection to all of this, and it deserves stating at full strength. Nobody competent bills from telemetry. Providers issue invoices and expose usage APIs; if you need per-tenant attribution, you put a gateway or proxy in front of the models and log every call to a durable store, the way you would for any metered resource. Telemetry is for latency and errors. Treating the trace as a ledger is a category error the standard never invited, and pointing at the standard for not being an accounting system is like blaming a thermometer for not being a scale.

I think that is right, and I think it is exactly why this is worth paying attention to. Because the invoice knows the total and nothing else. It cannot see which customer, which feature, which of your nine agents, which retry loop. That mapping exists in precisely one place — the call site, inside your process, at the moment the request is made — and the industry's standard way of emitting a fact from a call site is OpenTelemetry. So the demand is structural, not lazy. Teams reach for the trace because the trace is the only artefact that knows both the token count and the business context, and the conventions have obliged by putting gen_ai.conversation.id and gen_ai.agent.name next to the usage numbers. Everything needed for a chargeback is there. Everything except a promise that it will all arrive.

The conventions, for their part, keep reaching for money and then stopping short. Where a provider reports both consumed and billed token counts, instrumentation is told to report the billed one — the metric says MUST, the span says SHOULD — "so the value matches the units the customer is charged for." This is a specification that knows what its numbers are being used for. It is also a specification whose GenAI half is entirely at Development stability, where nothing is promised to survive, and whose most cost-relevant metric rework has been in review for eight weeks. All of which is in the project's own repository, which is worth saying plainly: this is a standard documenting its own limits in public, not a critic imputing them.

The architectural lesson is not about tokens. It is about what happens when a number crosses from one kind of system into another without anyone signing for it. Observability earned its freedom — to sample, to drop, to expire after thirty days, to change an attribute name in a minor release — by promising very little. Best-effort is not a weakness of the pipeline; it is the trade that made it cheap enough to instrument everything. A system of record makes the opposite trade: completeness, retention, an audit trail, someone who signs. The moment a value in the first kind of system is used to move money in the second, the freedom becomes a liability, and it does so silently. Nobody writes the ADR. There is no migration. There is just a dashboard that finance started trusting, and a sampling ratio that an SRE is still free to change on a Tuesday.

Most architects can tell you, without hesitating, which of their datastores is a source of truth. Ask them whether their traces are. The pause is the finding.