Term Kind Topic What it is
Backpressure in Streaming concept Streaming & Real-Time Data A slow downstream stage signalling upstream to slow down, so that queues stay bounded instead of consuming memory until the job dies.
Changelog Stream concept Stream-Table Duality A stream whose records are keyed updates, so replaying it from the beginning reconstructs a table — the same information in the other of its two forms.
Compaction Lag Cleaner Lag, Dirty Segment Lag concept Stream-Table Duality The delay between a key being overwritten and the superseded record actually being removed, which decides how much larger a compacted topic is than its key set and how long a rebuild from it takes.
Event Time Versus Processing Time concept Windowing The distinction between when something happened and when the system saw it, which determines whether results are reproducible.
Event Time vs Processing Time concept Windowing The distinction between when something happened in the world and when the system got round to handling it.
Exactly-Once Semantics EOS, Effectively-Once concept Exactly-Once Semantics The guarantee that each input record affects the output state precisely once, achieved through atomic state-and-offset commits rather than through delivering messages once.
Freshness Requirement concept Streaming vs Batch How stale data may be before the decision it supports degrades — the only question that justifies streaming over batch.
Idle Source Stall Watermark Stall, Silent Pipeline Halt, Quiet Partition Problem concept Streaming SLOs A stream processor whose event-time progress halts because one partition or source has stopped producing - so output ceases entirely while every component reports healthy.
Join Replay Misalignment One-Sided Replay, Split-Watermark Join concept Streaming Joins The failure that follows from replaying one input of a stateful join while the other stays live - event times no longer overlap, window state is never cleaned, and the output is wrong in both directions with n…
Key Skew concept Partition Keys & Ordering An uneven distribution of records across partitions, which caps throughput at the busiest partition regardless of how many exist.
Log Compaction concept CDC to Stream Retaining only the most recent value for each key in a topic, so the log becomes a durable snapshot of current state rather than a bounded window of history.
Partition Count Immutability Fixed Partition Count, Key-to-Partition Remap Hazard concept Partition Keys & Ordering The partition count of a keyed log is effectively permanent, because raising it changes which partition a key hashes to and therefore breaks per-key ordering and every table rebuilt by replaying that log.
Partition Key Skew Hot Partition, Key Skew concept Partition Keys & Ordering One partition receiving disproportionate traffic because a small number of key values dominate, capping throughput at what a single consumer can handle.
Poison Message concept Dead Letter Handling A record that a consumer cannot process and cannot skip, which halts its partition entirely until someone intervenes.
Provisional Result Restatement, Unsettled Aggregate concept Watermarks & Late Data An aggregate emitted before its window is definitively complete, which may be corrected when late events arrive - correct behaviour that destroys trust when it is not made visible.
Rebalance Cost Partition Reassignment Duration, State Restore Time concept Stateful Stream Processing The time a stateful consumer group is unavailable while partitions are reassigned and state restored - proportional to state size, incurred on every deployment, and invisible in throughput and lag metrics.
Replication Slot Retention Slot Lag, WAL Retention Pressure concept CDC to Stream The write-ahead log a database must keep because a change-data-capture consumer has not yet confirmed it, which turns a slow or stopped CDC pipeline into a disk-space failure on the primary database.
Reprocessing Window concept Backfill & Reprocessing How far back a pipeline can be replayed, set by the shortest retention anywhere along the path rather than by intent.
Schema Compatibility Mode concept Streaming Schema Evolution The rule stating which schema changes a registry will accept, which encodes whether producers or consumers are expected to upgrade first.
Semantic Schema Break Wire-Compatible Meaning Change concept Streaming Schema Evolution A change that keeps a field's wire format valid while changing what its values mean, so every compatibility check passes and every consumer computes a wrong answer without raising an error.
Serving Latency Budget concept Real-Time Serving Layer The end-to-end time from an event occurring to its effect being queryable, allocated across ingest, processing and serving.
Stateful Stream Processing concept Stateful Stream Processing Stream computations that must remember information across events — aggregations, joins, deduplication, sessionisation — and the durability machinery that makes them recoverable.
Stream-Table Duality concept Stream-Table Duality The equivalence between a stream of changes and a table of current state, where each can be derived from the other.
Streaming Cost Model concept Streaming Cost The cost structure of an always-on streaming system, where compute runs continuously regardless of volume and cross-zone data movement is a major line item.
Streaming Join concept Streaming Joins Combining two streams, or a stream and a table, where the records to be joined may not arrive at the same time or in the same order.
Table Bootstrap Lag Join Side Warm-Up, Lookup State Load Time concept Streaming Joins The interval after a stream job starts during which its lookup table is only partly loaded, so enrichment lookups miss for keys that appear late in the source topic and the job emits wrong output while looking…
Transaction Log Stream concept CDC to Stream Change events derived from a database's write-ahead log — faithful to the table's mutations and to its internal model rather than to the business's.
Unbounded Dataset concept Streaming & Real-Time Data Input with no known end, which removes the option of waiting for completeness and forces every aggregate to be provisional.
Watermark Event-Time Progress, Completeness Estimate, Lateness Boundary concept Watermarks & Late Data A heuristic assertion that no further events earlier than a given event time are expected, which is how a stream processor decides when a window is complete - and therefore where the correctness-versus-promptn…
Watermark concept Watermarks & Late Data The stream processor's assertion that no further events older than a given event time are expected, which is what allows a window to close.
Windowing Strategy concept Windowing The choice of how a continuous stream is divided into finite groups for aggregation, which determines both the meaning of the result and the state it requires.