Cache Stampede

The mechanism is simple and the effect is disproportionate. A cached value serving 5,000 requests per second expires. In the moment before it is repopulated, every one of those requests misses and goes to the origin, which now receives 5,000 concurrent copies of a query it was seeing once per TTL. The origin saturates, responses slow, so more requests pile up, and the cache cannot be repopulated because the query that would populate it is now queued behind thousands of duplicates.

The mitigations, each useful in different circumstances. Request coalescing — the first miss acquires a lock and fetches while the others wait for its result — is the most direct fix and turns 5,000 origin calls into one. Probabilistic early expiry has requests refresh the entry slightly before it expires, with randomness so only a few do, avoiding the synchronised miss entirely. Stale-while-revalidate serves the expired value while refreshing in the background, so users never wait and the origin sees one request. And TTL jitter prevents a whole class of entries populated together from expiring together.

The related and more severe failure is the cold cache after a restart or flush, where nothing is cached and the origin receives full traffic. Systems that cannot serve their traffic without a warm cache have a hidden dependency that will be discovered during a cache incident, and warming deliberately before taking traffic is the countermeasure.