Search the practice set
275 questions, 991 terms and 600 topics in 30 areas.
46 results for “Stateful Stream Processing”
Checkpoint Interval
How often a stateful processor persists its state and offsets, which trades steady-state overhead against how much work is redone after a failure.
Keyed State Size
The total state a job holds per key across all keys, which governs memory, checkpoint duration and recovery time.
Offset Management
How a consumer records its position in a stream, and the decision that determines whether processing is at-least-once or at-most-once.
Reactive Streams
A specification for asynchronous stream processing in which the consumer requests a specific number of items, making backpressure part of the protocol rather than an afterthought.
Changelog Stream
A stream whose records are keyed updates, so replaying it from the beginning reconstructs a table — the same information in the other of its two forms.
Event Stream
An append-only, retained log of events that many independent consumers read at their own position, and can re-read.
Event Stream Versioning
Transforming an old event version into the current shape when it is read, allowing a stored event history to be interpreted by evolved code.
Event Time Versus Processing Time
The distinction between when something happened and when the system saw it, which determines whether results are reproducible.
In-Country Processing
Keeping data within a jurisdiction across every path it takes — including backups, logs, telemetry, support access and the disaster recovery region.
Stateful Packet Filtering
Filtering that tracks connection state, so return traffic for an allowed outbound connection is permitted automatically.
Stream Enrichment
Attaching reference data to a stream by lookup against a materialised table rather than by joining two unbounded streams.
Stream-Table Duality
The equivalence between a stream of changes and a table of current state — each can be derived from the other.
Transaction Log Stream
Change events derived from a database's write-ahead log — faithful to the table's mutations and to its internal model rather than to the business's.
Value Stream
The end-to-end sequence of activities that delivers a result to a customer, viewed across whatever departments and systems it happens to cross.
Block Storage
A virtual disk attached to one instance at a time, presented as raw blocks and formatted with a filesystem — the storage databases and stateful workloads run on.
Change Data Capture
Publishing a stream of a database's row-level changes by reading its replication log, without modifying the application that owns it.
Consent Management
Capturing, storing, honouring and evidencing a data subject's permissions for specific processing purposes, including withdrawal.
Consumer Group
A set of consumers that cooperatively read one stream, with each partition assigned to exactly one member, so the group collectively processes every message once.
Horizontal vs Vertical Scaling
Adding more machines versus making one machine bigger — and the fact that vertical is underrated for stateful tiers.
Log-Based Ingestion
Building the pipeline around a database's own change log — an initial snapshot followed by a continuous delta stream, with the two stitched together.
Mainframe Offload
Moving read traffic and selected processing off a mainframe onto modern platforms, reducing cost and unblocking new development without replacing the core.
Materialized View
A precomputed, stored result of a query, refreshed on a schedule or from a change stream, read instead of recomputing.
Pipes and Filters
Decomposing processing into independent steps connected by channels, each transforming its input and passing it on.
Replay Pipeline
A single processing path that produces both live and historical results by re-running the same code over retained input, replacing the two-path Lambda arrangement.
Retrieval Pipeline Stages
The stages that turn a user question into grounded context — query processing, retrieval, reranking and assembly — each independently tunable.
Security Group
A stateful, instance-level firewall that allows specified traffic and denies everything else by default.
Serving Latency Budget
The end-to-end time from an event occurring to its effect being queryable, allocated across ingest, processing and serving.
TCP/IP
The layered protocol suite underneath essentially all application traffic — IP routes packets, TCP turns them into a reliable ordered stream.
Windowing
Grouping an unbounded stream into finite chunks so aggregation can produce results, defined over event time rather than arrival time.
A team proposes exposing their service's database change stream via CDC so other teams can consume it, avoiding the work of building an event API. What is your assessment?
Name what is actually being proposed The proposal is to publish the service's internal schema as its integration contract. CDC does not emit domain events; it e
A vendor claims their streaming platform provides exactly-once processing. How do you evaluate the claim?
Ask where the guarantee ends Nearly always at the platform's boundary. Within it, state and offsets commit together, so internal state reflects each input once.
Delivery feels slow and leadership wants a productivity initiative. You propose mapping the value stream first. What will you map and what do you expect to find?
Map every step from request to delivered change, with two numbers each Work time — how long the step actually takes when someone is doing it. Wait time — how lo
For each of these, choose a queue or a stream and justify it — order fulfilment tasks, an audit trail, cache invalidation, and rebuilding a search index.
Order fulfilment tasks — queue Each task is consumed once by one worker and is then irrelevant. Nothing re reads it; no second consumer needs the same task. Wha
Stateful Stream Processing
Keyed state, state backends, checkpoint size, and the restore time that follows.
Stream Processing Frameworks
Flink, Kafka Streams, Spark Structured Streaming — state, checkpointing and recovery.
CDC to Stream
Turning database changes into an event log, and how that differs from a domain event.
Stream-Table Duality
A changelog and a table as two views of the same thing, and materialising between them.
CDC Pipeline Design
Building on a change stream: snapshot plus delta, tombstones, and merge into the target.
Change Data Capture
Turning a database's replication log into a stream, and its coupling risk.
Firewalls & Security Groups
Default-deny, stateful rules, and restricting egress as well as ingress.
Horizontal vs Vertical Scaling
Scale out for stateless, scale up first for stateful.
Platform Team Topologies
Stream-aligned, enabling, complicated-subsystem and platform teams, and their interactions.
Streaming & Real-Time Data
General material on continuous processing of unbounded data.
Team Topologies
Stream-aligned, platform, enabling and complicated-subsystem teams.