Change Data Capture Pipeline  ·  View 16 of 21  ·  Operations

Deployment Architecture

One write region, a stateless capture standby, and the two stores that are deliberately multi-region.

Editable source SVG draw.io All views
Google Cloud europe-west1 — active Capture (regional managed) Datastream one stream per DB Snapshot workers Dataflow · 3 zones Transport Pub/Sub regional · 7 days Dead-letter topic Projection and control Dataflow appliers one job per sink Cloud Run operator API Spanner regional · control state Data residency and durability Multi-region EU BigQuery EU multi-region Dual-region archive Cloud Storage eur4 · 13 months europe-west4 — capture standby (RTO 30 min) Datastream standby resumes at last LSN Applier templates deployed, scaled to zero Cloud SQL for PostgreSQL HA, zonal failover Search index managed, regional replication slot by offset MERGE last committed LSN Deployment — Regional, With a Capture Standby Interface / broker Application we own Queue / topic Data store External / third party synchronous event / async One write region. The standby holds no state of its own: it resumes from the position in Spanner. Archive and metrics edges omitted. v 1.0 · owner Data Platform Architecture · date 2026-10

Decisions

  • Single capture region. Active-active capture would mean two readers of one replication slot, which is a correctness problem rather than a capacity one (ADR-15).
  • The standby holds no state: it resumes from the last committed log position in Spanner, which is why control state is regional-durable and separate.
  • BigQuery in EU multi-region and the archive dual-region, because the derived data is where residency obligations are actually read.

Numbers

  • RTO 5 minutes in-region, 30 minutes cross-region; RPO 0 relative to the source log position (assumptions).
  • Capture availability ≥ 99.9% monthly per source stream; control plane ≥ 99.5%.

Risks

  • A source failover changes the log-position timeline; comparing positions across it is the subtlest correctness trap in the design (ADR-15, view 21).
  • The standby is exercised only if it is exercised: an untested failover is an assumption, not an RTO.
◀  Schema Drift — Four Paths All views Observability  ▶
Solution Architecture v1.0 · Google Cloud · Data Platform Architecture · 2026-10 · 21 views · 15 architecture decision records16-deployment-architecture