LLM Rate Limiting & Traffic Management Service  ·  View 08 of 24  ·  Data

Policy and Usage Data Model

How a limit at any level in the hierarchy is expressed without a schema change, and what is recorded about a completed request.

Editable source SVG draw.io All views
organisation
org_id PK
name
tier ENUM(ENT,PRO,FREE)
fail_mode ENUM(OPEN,LOCAL,CLOSED)
budget_usd_month
organisation...
team
team_id PK
org_id FK -> organisation
name
cost_centre
team...
principal
principal_id PK
team_id FK -> team
subject (OIDC sub)
api_key_hash
priority
principal...
provider
provider_id PK
name
region
global_rpm
global_tpm
health_state
provider...
model
model_id PK
provider_id FK -> provider
name
context_window
price_in_per_1k
price_out_per_1k
model...
policy
policy_id PK
scope_type ENUM(ORG,TEAM,USER,MODEL,PROVIDER)
scope_id
model_id FK -> model NULL
rpm
input_tpm
output_tpm
max_concurrency
algorithm ENUM(BUCKET,SLIDING)
version
effective_from
policy...
reservation
request_id PK
principal_id FK -> principal
model_id FK -> model
reserved_input
reserved_output
slot_held
expires_at
state ENUM(HELD,COMMITTED,SWEPT)
reservation...
decision_audit
decision_id PK
request_id
verdict ENUM(ALLOW,REJECT)
reason_code
limiting_scope
policy_version
decided_at
decision_audit...
usage_event
event_id PK
request_id FK -> reservation
input_tokens
output_tokens
latency_ms
status
cost_usd
occurred_at
usage_event...
budget_ledger
org_id FK -> organisation
period (month)
spend_usd
forecast_usd
updated_at
budget_ledger...
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
0 : N
0 : N
1 : N
1 : N
1 : 1
1 : 1
N : 1
N : 1
1 : N
1 : N
Policy and Usage Data Model
Policy and Usage Data Model
One policy row per scope, so a limit at any level is added or removed without a schema change. No prompt or completion text appears anywhere in this model.
One policy row per scope, so a limit at any level is added or removed without a schema change. No prompt or completion text appears anywhere in this model.
v 1.0 · owner Data & AI Global Practice
v 1.0 · owner Data & AI Global Practice
Text is not SVG - cannot display

Decisions

  • One policy row per scope, keyed by scope_type and scope_id, with model_id nullable. Adding a limit at a new level, or a model-specific override, is an insert rather than a migration (FR5, FR6, FR8).
  • Every policy row carries a version and an effective_from. The version is what the limiter reports as a metric, which is how policy propagation delay becomes measurable rather than assumed.
  • reservation and usage_event are separate entities sharing request_id. The reservation is short-lived operational state; the usage event is the immutable accounting fact. Conflating them would put a 400-day retention requirement on hot state.

What is not here

  • No prompt text, no completion text, no message content of any kind. The model has no column that could hold one.
  • decision_audit stores the reason code and the limiting scope, not the request. It is what answers 'why was I throttled at 14:03' without retaining anything sensitive.
  • Provider credentials live in Vault and are referenced by path, never stored in this schema.

Retention

  • usage_event online in ClickHouse for 400 days, then Iceberg on MinIO for 7 years for audit.
  • decision_audit online for 90 days — long enough for a billing dispute, short enough to keep the table cheap.
  • reservation rows exist only in Valkey and are never written to PostgreSQL; the entity is shown here to make the request_id join explicit.