advanced 3 min answer

A team plans to publish database row changes via CDC as the organisation's domain events. What is wrong with that and what would you propose?

cdcoutboxdomain-eventscouplingnetflix
Show the full answer Hide the answer

What is being tested

Whether you can distinguish data movement from domain semantics, and whether you see the coupling being created.

What is wrong

1. Row changes are not business facts. A row in orders changed from status 3 to status 7. Was that a cancellation, a fraud hold, a refund, or a correction by a support agent? Consumers must infer intent from column diffs, and they will infer it differently from each other and incorrectly at least once. The producer knows the intent at the moment of the write; CDC discards it.

2. The internal schema becomes a public contract. Every consumer now depends on your table and column names. You can no longer rename a column, split a table, or change a representation without breaking systems you may not be able to enumerate. This is the most common long-term regret with CDC used this way, and it arrives quietly — nobody notices until the first refactor is blocked.

3. Granularity mismatch. A single business operation may touch six tables in one transaction. Consumers receive six separate change events with no indication that they belong together, and must reassemble the operation by correlating transaction IDs and timestamps.

4. No versioning story. Domain events can be versioned and evolved with compatibility rules. Table schemas evolve for storage reasons on a completely different cadence, driven by different concerns.

What to propose instead

A transactional outbox, captured by CDC. This is the design that gets the benefits of both:

  • The application writes a deliberately designed domain eventOrderCancelled, with the reason, the actor, and the fields consumers need — into an outbox table, in the same transaction as the business change. Atomicity is preserved; there is no dual-write gap.
  • CDC tails the outbox table and publishes. You keep the low-latency, complete, no-polling capture mechanism.
  • The published contract is the event schema, which you version and govern, not your internal tables, which you remain free to refactor.

Where CDC is genuinely the right tool

  • Data movement into warehouses, lakes and search indexes, where the consumer wants the data rather than the meaning.
  • Strangler migrations, where a legacy system cannot be modified to publish events and CDC is the only way to get its changes out without touching it.
  • Cache invalidation driven by actual changes rather than by the application remembering.
  • Auditing what actually happened, including changes made by batch jobs, admin tools and manual fixes that application-level publishing would miss. This completeness is CDC's real superpower.

Operational points worth raising

Bootstrapping must not lock the source. Every new consumer, every reprocessing after a bug, and some schema changes require an initial state dump. A design that takes a lock or a disruptive snapshot to do this discourages exactly the flexibility CDC exists to provide — which is why log-based capture work at Netflix and elsewhere focuses specifically on non-locking snapshots interleaved with the ongoing stream.

An unconsumed replication slot fills the disk. If a CDC consumer stops, the database retains log segments indefinitely, and eventually the source database goes down. An analytics pipeline taking down production is a spectacular and entirely avoidable failure, and it needs an alert on slot lag.