LLM Rate Limiting & Traffic Management Service  ·  View 22 of 24  ·  Assurance

Identity and Tenant Isolation

How a tenant is proved, and why one tenant cannot spend another's quota even if it asks to.

Editable source SVG draw.io All views
Client App
Client App
Keycloak
Keycloak
Envoy Gateway
Envoy Gateway
limiterd
limiterd
llm-gateway
llm-gateway
Vault
Vault
Provider
Provider
1. client_credentials, tenant realm
1. client_credentials, tenant realm
2. JWT: org, team, sub, tier
2. JWT: org, team, sub, tier
3. request + Bearer token
3. request + Bearer token
4. verify signature, cached JWKS
4. verify signature, cached JWKS
5. ext_authz Check over mTLS
5. ext_authz Check over mTLS
6. scope keys from claims, never from body
6. scope keys from claims, never from body
7. ALLOW + request_id
7. ALLOW + request_id
8. forward, claims in headers
8. forward, claims in headers
9. lease provider key for this tenant
9. lease provider key for this tenant
10. credential, TTL 15 min
10. credential, TTL 15 min
11. upstream call
11. upstream call
12. completion + usage
12. completion + usage
13. response, prompt not persisted
13. response, prompt not persisted
14. 401 on tenant mismatch
14. 401 on tenant mismatch
Identity and Tenant Isolation — proving who is asking
Identity and Tenant Isolation — proving who is asking
A tenant cannot spend another tenant's quota because the scope key is derived from signed claims, not from any field the caller supplies in the request body.
A tenant cannot spend another tenant's quota because the scope key is derived from signed claims, not from any field the caller supplies in the request body.
v 1.0 · owner Data & AI Global Practice
v 1.0 · owner Data & AI Global Practice
Text is not SVG - cannot display

The isolation property

  • Scope keys are derived from signed JWT claims, never from any field the caller supplies in the request body. A caller that sets tenant_id in the payload is ignored; the claim decides. That is what makes NFR7 enforceable rather than advisory.
  • The illustrative /v1/authorize payload in the brief carries tenant_id and user_id. In this design those are read from the token and the body fields are accepted only for logging, with a mismatch treated as a 401.
  • Keycloak realms are per tenant, so a token issued for one tenant is not merely unauthorised elsewhere — it fails signature-audience validation.

Credential handling

  • Provider API keys are leased per tenant from Vault with a 15-minute TTL. Rotation and revocation require no redeploy and no restart.
  • Client credentials tokens are 10-minute lived; JWKS is cached at the edge for 5 minutes with a forced refresh on unknown kid.
  • Nothing in the decision plane ever holds a provider credential — only llm-gateway does.

Risks

  • A compromised tenant credential lets an attacker consume that tenant's quota and budget. It cannot reach another tenant's, and the per-tenant budget cap bounds the financial loss.
  • Keycloak is on the critical path for new tokens. Its failure does not affect requests using an already-issued token, and the edge serves from cached JWKS for the cache lifetime.
  • Noisy-neighbour isolation is quota-based, not compute-isolated. A tenant within its quota that sends pathological request shapes still shares limiterd CPU; per-tenant concurrency caps bound this.