Event Time Versus Processing Time
The two clocks every streaming system has, why using the wrong one produces results that change on replay, and the skew that makes correctness a latency tradeoff.
A user taps a button at 23:58. Their phone is offline; the event uploads at 00:07 the next day. Which day's active-user count does it belong to? Answering "the day it arrived" is easy and gives a system whose historical numbers change every time you reprocess. Answering "the day it happened" is correct and requires a machinery that most of streaming's complexity exists to provide.
Two clocks
Event time is when the thing occurred, recorded in the event by the producer. It is a property of the data, so it is stable: reprocessing the same events assigns them to the same windows forever.
Processing time is when the pipeline saw the record. It is a property of the run, so it depends on network conditions, backlogs, restarts, and how fast the cluster happened to be. Two runs over identical input produce different results.
Anything that will be recomputed, compared across periods, or reconciled against a batch pipeline must use event time. Processing time is correct only for genuinely operational questions: current throughput, current lag, current error rate.
Skew and the completeness problem
The gap between the two is skew, and it is neither constant nor bounded. Mobile clients buffer while offline. Regions have different network quality. A consumer group that falls behind and catches up produces a burst of hours-old events.
This creates the defining problem of event-time processing. To emit the count for the window 23:00 to 00:00, you must decide that no more events for that window will arrive. You cannot know this; you can only estimate it. Waiting longer improves completeness and costs latency, and the choice between them is the fundamental streaming tradeoff, expressed through watermarks.
The consequence for architecture
Once results are keyed by event time, a stream and a batch job over the same data can agree, because both assign records to windows the same way. That is what makes the Kappa architecture viable: one event-time streaming pipeline, replayed from the log when you need to reprocess, rather than separate streaming and batch code paths computing the same numbers slightly differently. Lambda architectures existed largely because early streaming systems could not do event time correctly, and maintaining two implementations of every metric was the price.
When it breaks
Producer clocks are wrong. Event time comes from the client, and client clocks drift, are set incorrectly, and are occasionally adversarial. Events arrive timestamped in 1970 or in 2031, and a naive windowing implementation allocates state for the intervening windows and exhausts memory. Clamping timestamps to a sane range at ingestion is a standard defence that is usually added after the first incident.
There is often more than one event time. When the user tapped, when the client wrote the record, when the gateway received it, and when the domain event logically occurred can all differ and all be defensible. Choosing one per stream and documenting it prevents the situation where two teams join on time and silently mean different things.
Timezones and daylight saving. Windowing in local time means some days have 23 or 25 hours and one hour repeats. Store and process in UTC, convert at presentation, and treat any local-time computation in the pipeline as a bug.
Event time makes backfill semantics explicit and sometimes unwelcome. Replaying a month of data into an event-time pipeline correctly rewrites a month of windows. That is right, and it also means downstream consumers see historical numbers change, so they need to be built to tolerate restatement rather than assuming yesterday's aggregate is final.
12 flashcards for this concept
Click a card to reveal the answer.