Which parts of a grocery platform's data processing genuinely require streaming, and which are better as batch?

The decisive question

What decision does this data drive, and how quickly does the decision become wrong?

Streaming costs more to build, more to operate and more to reason about. It is justified when staleness has a measurable cost within the staleness window.

Genuinely streaming

• Inventory availability. Stale availability causes failed orders and substitutions with real cost, and it changes continuously. This is the clearest case.
• Order state and courier location, which drive live decisions and customer-visible tracking.
• Dispatch and matching, where both supply and demand change every few seconds.
• Fraud and abuse signals, where the decision must be made before the transaction completes.

Better as batch

• Catalogue and product attributes, which change daily and tolerate hours of lag.
• Analytics, reporting and business intelligence, where daily is the natural cadence of the decisions they inform.
• Model training and long-window features, where completeness matters more than latency.
• Financial reconciliation, where correctness and auditability dominate and a batch job with a clear boundary is easier to make correct.
• Anything whose consumers look at it once a day.

The mistake in each direction

Streaming everything means paying streaming's operational cost for data nobody reads sooner, and it makes correctness harder — exactly-once is unavailable, so every consumer must be idempotent, and reprocessing is a project rather than a rerun.

Batching everything makes the fast-changing data as slow as the slow-changing data. Merging catalogue and inventory into one nightly pipeline is the classic version of this, and it directly causes failed orders.

The property that decides the architecture

Separate pipelines by change rate and by decision latency, not by source system. Two feeds from the same partner with different freshness requirements are two pipelines — and giving them one SLO means the more demanding one is unmet or the less demanding one is over-engineered.