LLM Rate Limiting & Traffic Management Service  ·  View 09 of 24  ·  Data

Usage Accounting

How a completed request becomes a ledger row, a budget counter and eventually an invoice line, without ever being counted twice.

Editable source SVG draw.io All views
Emit
Emit
Decision Events
verdict + reason
Decision Events...
Completion Events
actual tokens
Completion Events...
Transport
Transport
Kafka usage.v1
24 parts · 7-day retain
Kafka usage.v1...
Schema Registry
Apicurio · Avro
Schema Registry...
Aggregate
Aggregate
Flink Windows
1 m / 1 h / 1 d
Flink Windows...
Cost Calculator
price × tokens
Cost Calculator...
Idempotent Upsert
keyed on request_id
Idempotent Upsert...
Store
Store
Usage Ledger
ClickHouse · 400 d
Usage Ledger...
Cost Marts
dbt · per org/team
Cost Marts...
Cold Archive
Iceberg on MinIO
Cold Archive...
Serve
Serve
Tenant Dashboards
Grafana
Tenant Dashboards...
Billing Export
signed monthly CSV
Billing Export...
Budget Enforcer
60 s refresh
Budget Enforcer...
budget counters back to L2
budget counters back to L2
Usage Accounting — from decision to invoice
Usage Accounting — from decision to invoice
Application we own
Application we own
Queue / topic
Queue / topic
Data store
Data store
Security / platform
Security / platform
Interface / broker
Interface / broker
event / async
event / async
At-least-once transport with an idempotent upsert keyed on request_id. Billing reads the ledger, never the stream, so a replay cannot double-charge a tenant.
At-least-once transport with an idempotent upsert keyed on request_id. Billing reads the ledger, never the stream, so a replay cannot double-charge a tenant.
v 1.0 · owner Data & AI Global Practice
v 1.0 · owner Data & AI Global Practice
Text is not SVG - cannot display

Decisions

  • At-least-once transport with an idempotent upsert keyed on request_id. Exactly-once delivery is not attempted; exactly-once effect is achieved at the sink, which is cheaper and easier to prove.
  • Billing reads the ledger, never the stream. A Kafka replay after an incident corrects the ledger and cannot double-charge a tenant.
  • The budget enforcer is a feedback loop back into L2 counters, closing the path from cost to admission (FR9, and the cost constraint in the problem statement).

Numbers

  • 24 partitions on usage.v1, keyed by org_id, sized for 1M events/s at the design ceiling with headroom for rebalancing.
  • Kafka retention 7 days — the replay window for ledger reconstruction.
  • End-to-end lag target under 5 minutes from completion to a queryable ledger row; budget counters refresh on a 60 s cycle.

Risks

  • Where a provider omits token counts on a streamed response, the row is marked estimated. Estimated rows are excluded from invoices and reported separately; the current share is under 2% of traffic.
  • Flink checkpoint failures stall aggregation without stalling ingest. The alert is on consumer lag, not on Flink health, because lag is what actually harms the tenant.