pattern

Kappa Architecture

Treating the stream as the single processing path, and handling historical reprocessing by replaying the log through the same code.

streamingarchitecturereprocessing

Kappa is defined by what it removes. Lambda architecture ran a batch layer and a speed layer in parallel — batch for correctness and completeness, streaming for latency — and reconciled the two at serving time. It worked, and the cost was maintaining every computation twice, in two frameworks, with two sets of subtle semantics that were expected to agree and periodically did not.

Kappa keeps only the streaming path. Reprocessing is not a separate system; it is replaying the retained log from an earlier offset through a new version of the same job, writing to a new output, and switching consumers when it catches up.

The requirements this imposes are the interesting part. The log must retain enough history to reprocess from, which for a year of history is a real storage decision — tiered storage made this far more affordable than it was when the idea was proposed. Jobs must be deterministic and side-effect free during replay, or reprocessing sends a decade of notifications. And event-time processing is mandatory, since replay compresses processing time arbitrarily.

The honest position today: Kappa is the right default for new systems, and many mature estates still run something Lambda-shaped because a warehouse already exists and doing large historical aggregations there is simply cheaper than replaying a stream.