Change Data Capture Pipeline  ·  View 05 of 21  ·  People and journeys

Journey — Ship a Schema Migration

Nobody tells the pipeline a migration is coming, so the design assumes it finds out from the stream.

Editable source SVG draw.io All views
Product engineer ships weekly Goal — Add a column and drop a dead one, on Tuesday, as usual Trigger — A feature needs a field; a deprecated field is finally unused Done when — Migration applied, analytics unbroken, nobody paged 1 · Write in the repo 2 · Apply 3 · Propagate ◆ moment of truth 4 · Find out ◆ moment of truth 5 · Resolve What they do Writes the migration Applies to the primary Reads the pause alert Confirms the drop What the platform does Nothing — no notice yet Decodes the DDL event Adds nullable column Evolves sink schema Pauses that one table Resumes from position How it feels Fine Uneasy Blamed Where it hurts No idea what replicates Alert arrives after apply Is a pause my outage? What answers it Replication list in CI Detection from the stream Compatible: automatic Blast radius: one table Resume, not re-snapshot Journey — Product Engineer Ships a Schema Migration Nobody tells the pipeline a migration is coming, so the design assumes it finds out from the stream. v 1.0 · owner Data Platform Architecture · date 2026-10

The finding

  • The emotional low is not the migration; it is being told afterwards that something downstream paused, and not knowing whether that counts as an outage.
  • That turns schema handling into two requirements: detection from the stream itself, and a blast radius of exactly one table (ADR-08).
  • A compatible change must propagate with no human in the loop, or engineers will route around replication by avoiding it.

Numbers

  • Compatible change propagates within 60 s; breaking change pauses that table within 10 s (assumptions).
  • A replication-list check in CI is the cheapest fix for the first phase's pain and is Phase 2 scope, not MVP.

Risks

  • A primary-key change looks compatible to a careless classifier and is not: it invalidates every key in the sink (view 15).
  • A dropped column that the pipeline silently ignores is the worst outcome of all — correct-looking and wrong.
◀  Journey — Wire Up a New Sink All views Journey — Chase a Number That Looks Wrong  ▶
Solution Architecture v1.0 · Google Cloud · Data Platform Architecture · 2026-10 · 21 views · 15 architecture decision records05-journey-ship-a-schema-migration