Search the practice set
275 questions, 991 terms and 600 topics in 30 areas.
58 results for “Stream-Table Duality”
Stream-Table Duality
The equivalence between a stream of changes and a table of current state — each can be derived from the other.
Changelog Stream
A stream whose records are keyed updates, so replaying it from the beginning reconstructs a table — the same information in the other of its two forms.
Stream Enrichment
Attaching reference data to a stream by lookup against a materialised table rather than by joining two unbounded streams.
Transaction Log Stream
Change events derived from a database's write-ahead log — faithful to the table's mutations and to its internal model rather than to the business's.
Event Stream
An append-only, retained log of events that many independent consumers read at their own position, and can re-read.
Event Stream Versioning
Transforming an old event version into the current shape when it is read, allowing a stored event history to be interpreted by evolved code.
Lakehouse Table Format
A metadata layer over object storage that adds atomic commits, snapshots and schema evolution to files that otherwise have none.
Nines Table
The mapping between availability percentages and permitted downtime, and the cost curve that comes with it.
Open Table Format
A metadata layer over files in object storage that supplies ACID transactions, schema evolution and time travel — the thing that turns a data lake into a lakehouse.
Route Table
The set of rules deciding where traffic leaving a subnet is sent, and the thing that actually makes a subnet public or private.
Value Stream
The end-to-end sequence of activities that delivers a result to a customer, viewed across whatever departments and systems it happens to cross.
Change Data Capture
Publishing a stream of a database's row-level changes by reading its replication log, without modifying the application that owns it.
Checkpoint Interval
How often a stateful processor persists its state and offsets, which trades steady-state overhead against how much work is redone after a failure.
Consumer Group
A set of consumers that cooperatively read one stream, with each partition assigned to exactly one member, so the group collectively processes every message once.
Covering Index
An index that contains every column a query needs, so the query is answered from the index without reading the table at all.
Data Lineage
A record of where each dataset came from, what transformed it, and what depends on it — traced at table and ideally column level.
Foreign Key Constraint
A database-enforced rule that a referencing value must exist in the referenced table — referential integrity that no application bug can violate.
Grain Declaration
Stating exactly what one row of a fact table represents, before any column is chosen, because every later decision depends on it.
Indexing Strategy
Choosing the set of indexes a table carries by working backwards from its actual queries, and accepting the write cost that each one adds.
Keyed State Size
The total state a job holds per key across all keys, which governs memory, checkpoint duration and recovery time.
Log-Based Ingestion
Building the pipeline around a database's own change log — an initial snapshot followed by a continuous delta stream, with the two stitched together.
Materialized View
A precomputed, stored result of a query, refreshed on a schedule or from a change stream, read instead of recomputing.
Offset Management
How a consumer records its position in a stream, and the decision that determines whether processing is at-least-once or at-most-once.
Operational Data Push
Sending modelled analytical data back into operational tools, which turns a warehouse table into a production dependency with none of the guarantees.
Outbox Pattern
Writing an outgoing message into a table in the same transaction as the business change, and relaying it to the broker separately, so the two cannot diverge.
Outbox Relay
The process that reads pending messages from an outbox table and publishes them to a broker, providing at-least-once delivery with no distributed transaction.
Predicate-Based Filtering
A policy that appends a filter to every query on a table based on who is asking, so one physical table serves many audiences safely.
Reactive Streams
A specification for asynchronous stream processing in which the consumer requests a specific number of items, making backpressure part of the protocol rather than an afterthought.
Route Propagation
Automatically inserting routes learned from a VPN or dedicated connection into a route table, rather than maintaining them by hand.
Schema Evolution
Changing a table's structure over time while keeping existing data readable and existing consumers working.
Source NAT
Rewriting the source address of outbound packets so that many private addresses share one public address, with a translation table mapping replies back.
Star Schema
A dimensional model with one central fact table of measurements surrounded by denormalised dimension tables describing them.
TCP/IP
The layered protocol suite underneath essentially all application traffic — IP routes packets, TCP turns them into a reliable ordered stream.
Time Travel
Querying a table as it existed at a previous version or timestamp, made possible by keeping the metadata and files of prior commits.
Windowing
Grouping an unbounded stream into finite chunks so aggregation can produce results, defined over event time rather than arrival time.
A dashboard has been wrong for three weeks. Nobody knows which upstream produced the table it reads. What is the governance failure, and what fixes it?
The failures, and there are three 1. No lineage. The question "which upstream produced this?" should be answerable in seconds by following a derived dependency
A dashboard query that took 200ms now takes 40 seconds. The table has grown to 200 million rows. Walk me through diagnosis and fix, including what you would not do.
What the interviewer is testing Whether you diagnose with evidence before changing anything, and whether you know the costs of the fixes you propose. Diagnosis,
A finance report double-counts revenue after a new fact table is added. What is the likely modelling error?
The likely error: a fan out join between fact tables at different grains The classic mechanism. You have an order lines fact at line grain and a shipments fact
A logic bug means six months of a derived table are wrong. You must reprocess without disrupting live consumers. How?
Establish the blast radius first Which downstream tables, reports and extracts consumed the wrong data, and did any of it leave the organisation — a regulatory
A table has 14 indexes and writes have become slow. How do you decide which to remove?
The approach 1. Get usage statistics, not opinions. Every major engine reports index scan counts — PostgreSQL's pg stat user indexes , SQL Server's sys.dm db in
A team proposes exposing their service's database change stream via CDC so other teams can consume it, avoiding the work of building an event API. What is your assessment?
Name what is actually being proposed The proposal is to publish the service's internal schema as its integration contract. CDC does not emit domain events; it e
A warehouse table now feeds the CRM through reverse ETL. It breaks, and the analytics team and the CRM team each say it is the other's problem. How do you resolve it?
The dispute is a design gap, not a personality problem A warehouse table built for analysis — nightly latency, best effort availability, schema changes made by
Delivery feels slow and leadership wants a productivity initiative. You propose mapping the value stream first. What will you map and what do you expect to find?
Map every step from request to delivered change, with two numbers each Work time — how long the step actually takes when someone is doing it. Wait time — how lo
For each of these, choose a queue or a stream and justify it — order fulfilment tasks, an audit trail, cache invalidation, and rebuilding a search index.
Order fulfilment tasks — queue Each task is consumed once by one worker and is then irrelevant. Nothing re reads it; no second consumer needs the same task. Wha
A team is choosing between Iceberg, Delta and Hudi for a new lakehouse. How would you approach the decision?
The formats are more alike than the debate suggests All three provide the same core: atomic commits over object storage, snapshots and time travel, schema evolu
Stream-Table Duality
A changelog and a table as two views of the same thing, and materialising between them.
CDC to Stream
Turning database changes into an event log, and how that differs from a domain event.
Open Table Formats
Iceberg, Delta and Hudi — transactions, snapshots and time travel over object storage.
Stateful Stream Processing
Keyed state, state backends, checkpoint size, and the restore time that follows.
Stream Processing Frameworks
Flink, Kafka Streams, Spark Structured Streaming — state, checkpointing and recovery.
Build vs Buy
Differentiation versus table stakes, priced over five years.
CDC Pipeline Design
Building on a change stream: snapshot plus delta, tombstones, and merge into the target.
Change Data Capture
Turning a database's replication log into a stream, and its coupling risk.
Platform Team Topologies
Stream-aligned, enabling, complicated-subsystem and platform teams, and their interactions.
Row & Column-Level Security
Restricting slices of a table rather than the whole table, and where it is enforced.
Team Topologies
Stream-aligned, platform, enabling and complicated-subsystem teams.