Three planes with different failure domains: data plane is regional and must survive alone; control plane is global and may be read-only for hours; usage plane is global and may lag by minutes.
limiterd and llm-gateway are separate deployables even though they are always co-deployed. They scale on different signals — decisions per second versus concurrent upstream connections — and llm-gateway holds long-lived streaming connections that must not force a limiter restart.
The Reservation Reaper is its own workload rather than a goroutine inside limiterd, so a rolling restart of the decision fleet cannot leave orphaned holds unswept.
Contracts
Envoy → limiterd: Envoy ext_authz v3 gRPC, 5 ms deadline, fail-open configured at the filter for HIGH-tier routes only.
limiterd → Valkey: EVALSHA against pinned script SHAs, 3 ms timeout, one retry to a replica.
policy.v1 is log-compacted, so a restarting pod replays only the current policy set and reaches steady state in about 1.2 s at 5,000 organisations.
Assumptions
60 limiterd pods per region at the 100k/s target, from a measured 18k decisions per second per 4-vCPU pod.
Valkey runs as 16 primary shards with one replica each, spread across three availability zones.
Health probes, metric scrapes and trace exports exist on every container but are omitted here — view 20 covers them.
