Real-Time Analytics Platform  ·  View 12 of 21  ·  Runtime

Windowing and Late-Arriving Events

Each window type against what state it keeps, when it emits, and what happens to an event that arrives after its window has closed.

Editable source SVG draw.io All views
Window definition
Window definition
State kept
State kept
Emit trigger
Emit trigger
Late within 2 min
Late within 2 min
Beyond watermark
Beyond watermark
Tumbling · 1 min
Tumbling · 1 min
Fixed 60 s buckets
Fixed 60 s buckets
One aggregate per key
One aggregate per key
Watermark passes end
Watermark passes end
Updated in place
Updated in place
Routed to late table
Routed to late table
Sliding · 5 min / 1 min
Sliding · 5 min / 1 min
Five overlapping windows
Five overlapping windows
5x state per key
5x state per key
Every 1 min slide
Every 1 min slide
Overlaps recomputed
Overlaps recomputed
Dropped and counted
Dropped and counted
Session · 30 min gap
Session · 30 min gap
Gap-based boundary
Gap-based boundary
Open session state
Open session state
Inactivity timeout
Inactivity timeout
Session extended
Session extended
New session opened
New session opened
Out-of-order handling
Out-of-order handling
event_time not ingest_time
event_time not ingest_time
Per-partition watermark
Per-partition watermark
Min across partitions
Min across partitions
Reordered in window
Reordered in window
Backfill job corrects
Backfill job corrects
Windowing and Late-Arriving Events
Windowing and Late-Arriving Events
Application we own
Application we own
Data store
Data store
Decision point
Decision point
Queue / topic
Queue / topic
Risk / gap
Risk / gap
Security / platform
Security / platform
Watermark lateness is 2 minutes; a slow partition holds the global watermark and is alerted on.
Watermark lateness is 2 minutes; a slow partition holds the global watermark and is alerted on.
v 1.0 · owner Streaming Engineering · date 2026-08
v 1.0 · owner Streaming Engineering · date 2026-08Text is not SVG - cannot display

Watermark choice

  • Lateness allowance is 2 minutes, chosen to cover mobile clients on intermittent connectivity
  • The global watermark is the minimum across partitions, so one slow partition holds everything
  • Watermark skew is alerted on before it becomes a freshness breach

State cost

  • Sliding windows hold five times the state of tumbling for the same key space
  • Session state is unbounded by definition; a 12-hour hard cap closes abandoned sessions
  • RocksDB state is checkpointed to ADLS so a restart does not rebuild from scratch

Correctness stance

  • Events beyond the watermark are routed to a late table, never silently discarded
  • The late table feeds a scheduled backfill that restates affected windows
  • Restated windows are published with a revision marker so consumers can detect the change
◀  Stream Processing Pipeline All views Event to Insight — Critical Path  ▶
Solution Architecture v1.0 · Azure · Data & AI Architecture · 2026-0812-windowing-late-events