Term Kind Topic What it is
Chunked Prefill Prefill Splitting, Interleaved Prefill, Piecewise Prompt Processing pattern AI Cost Management Breaking a long prompt's prefill into pieces interleaved with other sequences' decode steps, so that one enormous request cannot stall every in-flight interactive response.
Inference Cost per Interaction metric AI Cost Management The fully loaded token cost of one user interaction, including retrieval, retries, guardrails and agent loops, measured rather than estimated.
Token Cost Attribution practice AI Cost Management Assigning inference spend to features, tenants and users, so that cost can be managed by the people who influence it.