Streaming Aggregations for Real-Time Features
How to compute a sliding-window count over millions of entities within a serving latency budget, why exact windows are usually unaffordable, and the approximations that are safe.
"Number of card declines for this account in the last five minutes" is the kind of feature that decides whether a fraud model works. It is also the kind that cannot be precomputed on a schedule, cannot be recomputed at request time from raw events, and requires a streaming pipeline holding state for every entity in the system. The engineering here is about which approximations are acceptable.
Exact sliding windows are expensive
A true sliding window over five minutes, evaluated at request time, needs every event in that window for the entity. Storing raw events per entity and scanning them on each request means the online store holds unbounded data per key and the read cost scales with event volume, which is precisely wrong for a latency-critical path.
The standard approximation is tumbling sub-windows: maintain counts per one-minute bucket and sum the last five at read time. State per entity is five small integers rather than an event list, the read is a fixed cost, and the error is bounded by the sub-window size, since the oldest bucket is partially outside the true window. Finer buckets mean less error and more state, which is a dial rather than a decision.
Where the aggregation is additive and the window is long, exponentially weighted moving averages are cheaper still: one number per entity, updated in place, with an implicit soft window set by the decay rate. They cannot answer "in the last five minutes" precisely and they capture the same signal for many purposes, at constant state.
Approximate sketches
For aggregations that are not simple sums, probabilistic sketches make the state bounded.
Count-distinct over a window, such as distinct IP addresses per account, uses HyperLogLog: kilobytes of state per entity for a few percent relative error, against unbounded state for exactness, and the sketches merge, so sub-window buckets can be combined at read time exactly as counts can.
Quantiles use t-digest or KLL sketches. Heavy hitters use Count-Min Sketch or Space-Saving. In every case the question to answer before adopting one is whether the model's decision is sensitive to the error, and for the great majority of features whose value is bucketed or fed through a tree, a few percent error is immaterial.
When it breaks
Key cardinality decides feasibility. State scales with the number of active entities, so a feature keyed by user is bounded by user count and a feature keyed by user-and-item pair is bounded by their product. The second is regularly proposed and is regularly the reason a streaming feature pipeline cannot be operated.
State TTL is mandatory. Without expiry for inactive entities, state grows monotonically until the job fails on a checkpoint that no longer fits. Setting TTL correctly requires knowing the entity activity distribution, and the long tail of rarely-active entities is usually much larger than expected.
Restarts must rebuild state. A pipeline restarting from an old checkpoint must replay to catch up, and during that replay it produces feature values computed from partial windows. Serving those values is worse than serving nothing, so the serving path needs a way to know the pipeline is not caught up, which means exposing lag as part of the feature's metadata.
Offline reproduction of an approximate feature is genuinely hard. Training data must contain the same approximate values the online path produced, including its bucket boundaries and sketch error. Recomputing exact values offline creates skew in the direction that is hardest to detect: the offline feature is better than the online one, so the model learns to rely on precision it will not have.
14 flashcards for this concept
Click a card to reveal the answer.