Spot Capacity Orchestrator  ·  View 25 of 32  ·  6 · Operations

Deployment Architecture

One region's control plane spread over three halls on guaranteed capacity, reclaimable pools and the floor across all four, and the shared estate services it rides on.

Editable source SVG draw.io All views
Region r1 · r2 and r3 are identical and independent Hall A sco-controller leader · partitions 1–4 PostgreSQL primary Patroni Reclaimable pools 9 SKUs × 4 sizes Hall B sco-controller standby PostgreSQL replica synchronous Reclaimable pools 9 SKUs × 4 sizes Hall C sco-controller standby PostgreSQL replica asynchronous Reclaimable pools 9 SKUs × 4 sizes Hall D Guaranteed floor spread over 3+ halls Reclaimable pools 9 SKUs × 4 sizes Estate data services Kafka · ClickHouse · Ceph Prometheus and Thanos per region · global query Forgejo and CI mirrored to r2 sync commit streaming Deployment — One Region of Three, Four Halls Application we own Data store External / third party Security / platform synchronous event / async The management cluster spans halls A to C on guaranteed capacity. Losing any one hall leaves a leader, a synchronous replica and the whole node plane working. v 1.0 · owner Compute Platform Architecture · date 2026-09

Decisions

  • The control plane never runs on reclaimable capacity. A management Kubernetes cluster on guaranteed capacity in halls A to C holds three controller replicas and a Patroni PostgreSQL cluster with one synchronous and one asynchronous replica.
  • Regions are independent: no cross-region call on any control or notice path. Only the Git repository and CI are shared, mirrored to a second region.
  • The guaranteed floor for each tier-1 workload is spread over at least three halls with no hall holding more than 40% of it.

Footprint and cost

  • Per region: 3 controller pods at 2 vCPU, 3 PostgreSQL nodes at 8 vCPU, scorer and watcher at 4 vCPU, plus shares of Kafka, ClickHouse and Prometheus. About 145 vCPU equivalent across three regions, including agents at steady state, which at the guaranteed rate is the €3.2k a month on view 28.

Hall loss

  • Losing a hall loses up to 40% of a workload's floor for as long as it takes the floor keeper to rebuild it elsewhere from committed capacity: 15 minutes at most, stated as a risk rather than bought away with a 40% larger floor.