AI Code Completion Service · View 17 of 21 · Operations
Decisions
- Prefix caches are never replicated across regions: they are recomputable, and replicating them would couple regions for no durability gain.
- Policy and seats have one write region with regional read replicas; the model registry is geo-replicated because weights are large, immutable and read everywhere.
- Measurement is pinned to a third zone so that a telemetry incident and a serving incident are not the same incident.
Assumptions
- 4,000 accelerators at peak across three regions; a region loss costs cold starts, not correctness.
- RTO 5 minutes to serve the suggestion path from another region; the sovereign dedicated-pool option is phase 3 and is drawn as out of scope.
Risks
- Three regional fleets batch less efficiently than one global fleet. This is the cost of the latency contract, and it is the trade the eighth core architecture question refuses to settle for the reader.
