Token Accounting and Cost Attribution
Why a single provider invoice cannot be allocated to teams, features or customers without instrumentation, and the tagging discipline that makes AI spend attributable.
A monthly bill arrives showing total spend by model. It cannot be allocated to a feature, a team, a customer or a request path, because nothing in the request carried that information. Without attribution, the organisation has one number it cannot act on, and every conversation about reducing it is a guess.
What has to be recorded
Attribution has to happen at the point of the call, because nothing downstream can reconstruct it. Every model invocation should record the model and version, input and output token counts as reported by the provider, the feature or code path that made it, the team or cost centre owning that path, the customer or tenant if the product is multi-tenant, and the request or trace ID linking it to the rest of the system.
Token counts should come from the provider's response rather than from local estimation. Local tokenisers drift from the provider's, special tokens and formatting differ, and cached-token accounting is not something a client can compute.
With those fields, spend becomes a queryable fact table joinable to product analytics, and the questions people actually ask, which feature costs most per active user, which customers are unprofitable, what a proposed change would save, become answerable.
The dimensions worth having
By feature drives product decisions, since it exposes an expensive feature with low engagement.
By customer or tenant drives pricing, and in a usage-heterogeneous product it reliably reveals a small number of accounts consuming a disproportionate share.
By model shows the mix and quantifies what routing changes would save.
By cached versus uncached tokens shows whether the prompt caching that was implemented is actually hitting, which is frequently not the case and is invisible without the split.
Over time shows drift, and the most common finding is a system prompt or a retrieval configuration that has grown without anyone deciding to grow it.
When it breaks
Shared infrastructure resists allocation. A self-hosted model serving several teams produces a GPU bill rather than a token bill, and dividing it by request counts ignores that requests differ enormously in length. Weighting by tokens or by GPU-seconds consumed is defensible; dividing by request count is not, and it systematically subsidises the expensive callers.
Attribution tags rot. Feature names in code drift from feature names in the product, teams reorganise, and cost centres change. Without validation that every call's tag resolves to something current, an increasing share of spend lands in an "unknown" bucket that nobody owns.
Chargeback changes behaviour, sometimes badly. Making teams pay for their AI spend does reduce it, and it also discourages experimentation and pushes teams toward cheaper models regardless of quality. Showback, reporting cost without billing it, gets most of the awareness benefit with fewer distortions.
The cheapest request is the one not made. Attribution surfaces where the calls come from, and the largest savings often come from removing a call entirely: caching a deterministic result, using a rule where a model was unnecessary, or not calling on a path where the output is discarded. Optimising the cost of a call that should not exist is a common local optimum.
12 flashcards for this concept
Click a card to reveal the answer.