Distributed Systems 02 Sep 2026 26 min read

A cache miss is a protocol, not an event

How production systems stop a cache miss from becoming an outage: who recomputes, who waits, and what the waiters get.

Reconstructs the miss protocol behind every production cache from Facebook, Slack and Wikimedia postmortems and the implementations at Google, Instagram, Discord, DoorDash, Netflix, memcached, Rails, nginx, Varnish and Fastly. After reading, an architect can name the three primitives every stampede defence composes (elect one recomputer, serve stale to the rest, spread the expiry edge), pick the right layer for each, and defend the choice with measured numbers.

The finding that surprised me

In the worst published cache outages the stampede was triggered by the machinery that manages the cache, not by an expiring key: Facebook's repair path deleted keys on every database error, and Slack's cache manager flushed nodes at daily peak; meanwhile a 52% hit ratio, not zero, was enough to multiply Wikimedia's site latency five-fold.

What you get out of it

  • Every stampede defence composes three primitives: elect one recomputer (lease, lock, singleflight), give waiters a stale answer (SWR, race_condition_ttl, interim values), and spread the expiry edge (jitter, XFetch).
  • The repair path is a miss path: Facebook 2010 shows delete-on-error becoming a self-sustaining stampede that outlived its root cause.
  • The cliff is above zero: Wikimedia's incident ran at a 52% hit ratio, and origin capacity must be planned for the worst tolerated ratio, not for average.
  • Coalescing needs an answer for unshareable responses: Varnish documents its waiting list degrading into one-by-one request serialization when the response is uncacheable.
  • Facebook's 2013 lease design is now in mainline memcached's meta protocol (win/stale/lost flags), so the strongest server-side defence no longer requires a fork.

Scope

Why this, now. Every cache product ships some stampede protection off by default, and the public incident record keeps showing the outage arriving through the cache's own management machinery rather than the textbook expiring key.

What it does not cover. Eviction and admission policy, CPU and browser caches, hardware cache coherence, and retry-side amplification, which has its own guide in this collection.

Open the field guide → Self-contained: it loads nothing at read time, follows your system theme, and prints cleanly.