Fan-out Threshold

Pushing an item to every recipient gives cheap reads and costs one write per recipient. Having recipients pull gives cheap writes and costs an expensive gather per read. Neither is right for all cases, because recipient counts in real systems span many orders of magnitude.

The fan-out threshold is the point at which the system switches strategy — and the interesting engineering is not the threshold itself but the machinery that keeps entities on the correct side of it.

Why a hybrid is necessary

For the overwhelming majority of publishers, push is correct: a handful of writes, and reads become a single sequential lookup.

For the small number with enormous recipient counts, push produces a write-amplification event where one action becomes millions of writes arriving as a burst — saturating the write path and delaying every other user's updates. Nothing failed; the design simply met the tail of its own distribution.

Pull for everyone would make reads expensive for the most engaged users, who follow the most sources.

Implementation patterns

• Classify by recipient count, with the threshold derived from measured write capacity rather than chosen by intuition.
• Monitor and reclassify. An entity that grows past the threshold must be reclassified automatically; a threshold with no reclassification is a latent incident.
• Merge at read — combine the materialised timeline with a bounded pull from the large publishers the reader follows.
• Consistent ordering across both sources, or the merged result visibly jumps.
• Skip inactive recipients. A large fraction have not opened the product in months; materialising for them is pure waste, and lazy materialisation removes it.
• Prioritise the fan-out — active and recently-online recipients first, with the long tail completing over minutes.
• Idempotent per-recipient writes, so a partially-completed fan-out can be retried without duplicating.
• Bounded materialised state, capped at a few hundred entries with older content fetched from source.

Industry example

The pattern recurs wherever one publisher reaches many subscribers: a post from a very large account, a message in an enormous community channel, a live-stream chat message visible to a hundred thousand concurrent viewers, or a notification triggered for millions after a major event.

Chat platforms hit it most sharply because latency expectations are seconds rather than minutes. A message in a three-person channel can reasonably be pushed to each member's connection; the same mechanism in a million-member channel would produce a million writes for one message typed by one person. So large channels invert — the message is written once and connected clients read from it — with identical user-visible behaviour and a completely different mechanism.

The same threshold logic governs presence: precise for small groups, sampled and approximate above a size where nobody can perceive the difference.

Failure scenarios

• A single strategy for all publishers, producing either write-amplification events or slow reads for engaged users.
• A threshold with no monitoring, so a growing entity crosses it unnoticed and causes the incident the hybrid existed to prevent.
• Ordering inconsistency at the merge, producing a visibly jumping result.
• Wholesale retry of a partial fan-out, duplicating for recipients who already received it.
• Unbounded materialised state, growing without limit for active recipients.

Trade-offs

The hybrid buys both benefits and costs two code paths, a classification rule that must be maintained, and a merge step that is a common source of subtle ordering bugs. For a system whose recipient counts are genuinely uniform, a single strategy is simpler and correct.

The judgement is whether the distribution has a meaningful tail — and in consumer social, messaging and notification systems it always does.

Interview question

"An account with forty million followers posts every few minutes. Walk me through what happens under fan-out-on-write, design the hybrid, and tell me how an account that grows past your threshold gets reclassified."