Prompt & Configuration Registry  ·  View 17 of 21  ·  Operations

Deployment Architecture

Where each plane runs, and why the thing the fleet touches is the smallest tier on the page.

Editable source SVG draw.io All views
Primary data centre Platform cluster — Kubernetes Config API 3 replicas Build controller Argo Workflows Release controllers leader-elected PostgreSQL Patroni, 3 nodes Distribution tier — the only tier the fleet touches etcd 3-node quorum OCI registry 2 replicas MinIO erasure-coded NATS JetStream 3-node Evidence tier Kafka 3 brokers ClickHouse 2 shards × 2 Audit WORM Object Lock Shared platform services Keycloak SPIRE server OpenBao HSM seal Prometheus + Loki Application clusters — where resolution actually happens Cluster A · ~180 services AI feature pod SDK + resident bundle Cluster B · ~120 services AI feature pod SDK + resident bundle Batch & offline Replay job pinned digest Secondary site read replica of artefacts Git remote pull by digest Deployment Architecture Application we own Data store Queue / topic Security / platform External / third party synchronous The pointer store's writable primary is single-site in the MVP; the secondary carries an artefact read replica, which is enough to keep the fleet serving. The 30 s poll backstop, the exposure path to Kafka and MinIO's replication are drawn on views 14, 15 and 10. v 1.0 · owner Platform Architecture · d 2026-09

Decisions

  • The distribution tier is drawn as a trust boundary of its own because it is the only tier ~300 application pods are allowed to reach. Everything else in the platform cluster is reachable by operators and by nothing else.
  • Resolution runs in the application clusters, inside pods this platform does not deploy. The registry has no presence there beyond a library and the bundle it holds.
  • The secondary site carries an artefact read replica and no writable pointer store. That is enough to keep the fleet serving and not enough to promote — a deliberate asymmetry (ADR-17).

Assumptions

  • Control-plane availability ≥ 99.9% monthly, resolution availability ≥ 99.99% measured at the calling service (stated assumptions). The second is not delivered by this diagram's redundancy; it is delivered by fail-static.
  • etcd at 3 nodes, NATS at 3, Kafka at 3 brokers, ClickHouse 2 shards × 2 replicas — sized for 200 promotions/day and 500 M exposures/day, not for the resolution rate, which never reaches this cluster.
  • RTO 15 minutes for the control plane; RTO 0 for resolution, because a control-plane outage is not a resolution event.

Risks

  • Single-site writable pointer store means a site loss freezes the fleet on its current digests until the secondary is promoted by hand. Accepted for the MVP and named as Phase 3 work — frozen is a much better failure than wrong.
  • The 30 s poll from every pod in two clusters is a steady read load on etcd that grows with the fleet. Past roughly 20,000 pods the poll has to move to the artefact store or to a fan-out tier.
◀  Degradation Paths All views The Release Loop  ▶
Solution Architecture v1.0 · open source, self-hosted on Kubernetes · Platform Architecture · 2026-09 · 21 views · 17 architecture decision records17-deployment-architecture