Streaming Cost Model
The cost structure of an always-on streaming system, where compute runs continuously regardless of volume and cross-zone data movement is a major line item.
Streaming's cost profile differs from batch in a way that catches teams out at budget time. Batch compute runs when there is work; streaming compute runs always, because latency is the product. A pipeline processing a thousand events a day costs nearly what one processing a million does, since both keep the same consumers resident.
The components worth modelling before committing. Persistent compute, sized for peak rather than average because backlog recovery requires headroom. Replication, since a durable log with a replication factor of three stores every byte three times and writes it across the network twice. Cross-zone network transfer, which is a genuinely large and frequently unnoticed line item in cloud deployments — consumers reading from a leader in another availability zone pay per gigabyte, continuously, forever. State storage for stateful jobs, plus its changelog. And retention, where a long replay window is exactly what makes Kappa-style reprocessing possible and is also a standing storage bill.
The design responses are concrete: rack-aware or zone-aware consumption to keep reads local, tiered storage so long retention lands on object storage rather than broker disk, compaction on keyed topics, and an honest look at whether each stream needs streaming at all. A material share of "real-time" pipelines serve dashboards that a human reads twice a day.