A dispatch analytics pipeline computes per-minute aggregates and the numbers keep changing after the fact. What is happening, and what decisions must be made explicit?

What is happening

Events are arriving after the window they belong to has been computed. A courier's device buffers while offline and uploads twenty minutes of positions at once; a mobile client retries after a network failure; a partition lags and its events arrive late.

Aggregating by event time means a window is never definitively complete, because more events for it may still arrive.

The decisions that must be explicit

• Event time or processing time? Event time gives correct answers and requires waiting; processing time gives immediate answers about when things were seen rather than when they happened. Conflating the two causes most time-related bugs in stream processing — store both.
• How long to wait, which is the watermark. A watermark is a bet: it declares that events older than this will be treated as late. Too aggressive and correct data is discarded; too conservative and results are slow.
• What happens to events arriving after the watermark: dropped, counted separately, or triggering a recomputation and an update. All three are defensible and leaving it undefined is not — the default is usually silent dropping.
• Whether downstream consumers can handle a restatement. If a window's value can change, every consumer — dashboard, alert, downstream aggregate — must tolerate it, and most are built assuming it will not.

Why the numbers change

Because the pipeline is emitting a provisional result and then correcting it, which is the right behaviour and is undocumented. A dashboard showing a value that changes without explanation destroys trust in the whole platform.

The fix is to make the provisionality visible: mark windows as provisional until the watermark passes, and show when a value was restated.

The design that avoids most of it

Separate the fast provisional path from the correct settled path. The live dashboard reads the provisional stream; the reporting layer reads the settled aggregates computed after the watermark.

Trying to serve both from one computation means either the live view is slow or the reports are wrong — which is the trade the separation removes.