What does stream-table duality mean practically, and how does it change system design?

What it means practically

A table is the accumulated result of a stream of changes; a stream is the sequence of changes to a table. Each can be derived from the other, which means they are two views of the same information rather than different things.

Concretely: a database's replication log is a stream, and replaying it reconstructs the table. A stream of updates aggregated by key produces a table of current values. A table's contents can be emitted as a stream of its rows.

How it changes design

1. Change data capture is not a special integration; it is reading the table's stream. That reframing makes the operational database's log the natural source for everything downstream, rather than a bolted-on extraction — and it explains why log-based CDC is superior to polling: it is reading the thing the database already maintains.

2. Materialised views become derived tables of a stream, maintained incrementally rather than recomputed. The read model is the accumulated stream, which makes freshness a function of lag rather than of refresh schedule.

3. Reprocessing is replaying. If a table is a function of its stream, correcting a bug means replaying the stream through corrected logic. This is what makes retained raw events valuable and what makes a transformation error recoverable rather than permanent.

4. Joins between a stream and a table are natural — an enrichment lookup is a join against the current state of another stream's accumulated table, which is how stream processors express reference-data enrichment.

Where the abstraction leaks

• Retention. Reconstructing a table by replay requires the stream to still exist. Bounded retention means the table can only be rebuilt from a snapshot plus the retained tail.
• Compaction changes the semantics. A log compacted to the latest value per key is no longer the full history — it reconstructs the current table and not the sequence of changes.
• Ordering. Reconstruction requires the changes in order, which is only guaranteed per partition key.
• Deletes need explicit representation, or replay reconstructs rows that were removed.

The practical value

It unifies the operational and analytical worlds. The same change stream feeds a replica, a search index, a cache, an analytical store and a real-time aggregate — one source of change, many derived views, each with its own lag and its own guarantees.

That is a simpler architecture than separate extraction mechanisms per consumer, and it is why the change log is the right integration point.