Fan-out on Write vs Fan-out on Read

Fan-out on write (push). When an item is published, write a copy into each follower's timeline. Reads are then a single sequential lookup — fast, cheap and simple. Writes cost one operation per follower.

Fan-out on read (pull). Store the item once. At read time, gather items from everyone the reader follows and merge them. Writes are trivial; reads are an expensive scatter-gather whose cost scales with how many sources the reader follows.

The choice is a straightforward trade between write amplification and read amplification — until the follower distribution is skewed, which it always is.

Why it matters

Read/write ratios in social and feed systems are extremely lopsided, which argues strongly for fan-out on write. But the follower distribution has an enormous tail: an account with tens of millions of followers turns one publish into tens of millions of writes, arriving as a burst. That single event can saturate the write path for everyone.

Implementation patterns

The mature answer is hybrid, and the details are where the engineering lives:

• Fan-out on write for ordinary accounts, which is the overwhelming majority of publishes.
• Fan-out on read for high-follower accounts. Their items are stored once and merged into a reader's timeline at read time. There are few such accounts, so the merge is bounded.
• Merge at read, combining the pre-materialised timeline with a small pull from the followed celebrity accounts.
• Skip inactive recipients. A large fraction of followers have not opened the product in months; writing to their timelines is pure waste. Materialise lazily for them instead.
• Asynchronous, prioritised fan-out. Active and recently-online followers first, so the people likely to look receive it quickly and the long tail completes over minutes.
• Bounded timelines. Cap materialised timelines at a few hundred entries; older content is fetched from source.

Industry example

The pattern recurs wherever one publisher reaches many subscribers: a post from a very large account, a message to an enormous community channel, a live-stream chat message visible to a hundred thousand concurrent viewers, or a notification triggered for millions of users after a major event.

Chat and community platforms hit it most sharply because latency expectations are seconds, not minutes. A message in a three-person channel can reasonably be pushed to each member's connection. The same mechanism in a million-member channel would produce a million writes for one message typed by one person. So large channels invert: the message is written once, and connected clients subscribe to the channel and read from it. The user-visible behaviour is identical; the mechanism is completely different, chosen by a threshold on membership size.

The same threshold logic governs presence: precise for small groups, sampled and approximate above a size where nobody can perceive the difference anyway.

Failure scenarios

• Pure fan-out on write with no celebrity handling — one publish causes a write-amplification event that delays every other user's timeline updates.
• Pure fan-out on read — timeline latency degrades for users following many accounts, which are usually the most engaged users.
• Hybrid with an unmonitored threshold, so accounts that grow past it are never reclassified.
• Fan-out that is retried wholesale. A failure partway through re-sends to recipients who already received it, requiring idempotent per-recipient writes.
• Ordering surprises at the merge. Pushed and pulled items must interleave by a consistent ordering key, or the timeline visibly jumps.

Trade-offs

Fan-out on write buys read latency with storage and write amplification, and makes deletion and permission changes expensive — revoking access means finding every copy. Fan-out on read buys cheap writes and correct-by-construction permissions with unpredictable read latency. The hybrid buys both benefits at the cost of two code paths, a classification rule that must be maintained, and a merge step that is a common source of subtle ordering bugs.

Interview question

"A user follows a celebrity account with 40 million followers who posts every few minutes. Walk me through what happens under fan-out on write, and design the hybrid — including how you decide the threshold and what happens to accounts that cross it."