What should drive the choice of stream processing framework for a high-volume recommendation platform?

What should drive it

1. State management. How much state, how it is checkpointed, how recovery works, and whether state can exceed memory. For recommendation features — session context, recent interactions, windowed counters — state is substantial, and a framework whose state model does not fit becomes the constraint.

2. Exactly-once state semantics, if aggregates must be correct across failures. Frameworks differ in whether this is available and what it costs in throughput and latency.

3. Latency versus throughput characteristics. Micro-batch and true streaming models sit at different points, and the requirement should decide rather than the fashion.

4. Backfill and reprocessing. Replaying months of history through the same logic as the live path is a routine operation. A framework where the batch and streaming paths are separate implementations reintroduces the divergence that causes training/serving skew.

5. Operability at your scale. Rebalancing behaviour, checkpoint duration, upgrade path, and what happens when a job must be restarted with changed state — which is the operation that hurts most in practice.

6. Team familiarity, which is undervalued: the best framework is frequently the one that can be debugged at 3 a.m.

The property most underweighted

How state schema changes are handled. A stateful job whose state layout changes cannot simply be redeployed — the existing state must be migrated or discarded, and discarding means recomputing from history.

For long-window state this is expensive, and it is discovered the first time a feature definition changes. Frameworks differ substantially here and it rarely appears in evaluations.

The architectural decision that matters more than the framework

One ingestion path, two consumption paths, with a feature store as the seam so a feature is defined once and executed by both the streaming and batch paths.

Duplicating ingestion produces two pipelines that disagree — the origin of training/serving skew, which is the most expensive bug class in recommendation systems because it degrades quality silently rather than failing.

No framework choice compensates for getting that wrong.