Fan-Out Amplification

A push system's workload is not the rate of events arriving. It is events multiplied by subscribers. One price tick delivered to a hundred thousand watchers is one input and a hundred thousand outputs, and the cost is in serialisation and system calls rather than in any business logic.

This changes what the system is limited by: connection count is the scaling unit, and capacity is consumed whether or not anyone is actively doing anything.

Why it matters

It means capacity planning cannot be derived from business volume. A trading platform's order rate might be modest while its price fan-out is enormous; a social platform's post rate is tiny compared with the delivery volume it produces. Sizing the broadcast tier from the write tier's numbers under-provisions it by orders of magnitude.

Implementation patterns

• Conflation. A subscriber that has fallen behind receives the latest value, not a backlog. This bounds work by entity count rather than by event rate, and it is the correct overload behaviour for superseding data.
• Subscription filtering at the edge, so a client receives only what it asked for rather than the whole feed filtered client-side.
• Time-window batching — accumulate for 50 to 100ms and send once — which reduces system calls by an order of magnitude at a latency cost that is invisible in the product.
• Binary encoding, because serialisation dominates at this multiplication factor and text formats cost several times more.
• A dedicated fan-out tier with its own capacity, scaling and failure domain.
• Jittered reconnection with headroom, because when a fan-out node dies all its connections reconnect simultaneously to nodes that are now under-provisioned. Without jitter, one node failure cascades.

Industry example

A brokerage such as Zerodha pushes live prices to very large numbers of concurrent connections while simultaneously accepting orders. The two workloads have opposite overload behaviours: prices must drop and conflate under pressure, orders must never be lost. That incompatibility is the reason they cannot share a connection pool, a process or a queue — and the classic incident is not a trading system failing under trading load, but failing because everyone opened the app at once and read traffic exhausted a resource the order path needed.

The same shape governs live-class platforms, sports scores, collaborative editing presence, and any notification fan-out.

Failure scenarios

• Broadcast and critical writes sharing infrastructure, so the burst starves the writes.
• Per-message sends with no batching, making system calls the bottleneck well before bandwidth.
• No conflation, so a slow subscriber accumulates an unbounded backlog and consumes memory until the node fails.
• Synchronised reconnect storms after a node failure, cascading across the tier.
• Capacity planned from event rate rather than from event rate times subscribers.

Trade-offs

Conflation loses intermediate values, which is correct for superseding data and unacceptable for anything where each event matters. Batching adds latency. Edge filtering requires the fan-out tier to hold subscription state, which complicates failover.

The alternative — treating every subscriber delivery as an independent reliable message — is correct for notifications with individual value and ruinously expensive for continuous streams. The classification of the data as superseding or non-superseding is the decision everything else follows from, and it must be explicit rather than a property of whichever transport was chosen.

Interview question

"Your price feed serves 200,000 concurrent connections and your order rate is 500 per second. Size both tiers and tell me what happens to each one when a fan-out node dies."