Which caching failure appears only under load, and what are the fixes in order of deployability?

What is being tested

Whether you know the failure that is invisible in testing and obvious in production at peak.

The failure: the stampede

Cache-aside logic contains an unstated assumption — that only one caller will miss at a time.

At peak, when a hot key expires, a thousand callers miss within the same few milliseconds and every one independently queries the source. The database was comfortably serving the 0.1% that normally missed; it cannot serve 100%.

It is invisible in a load test with a warm cache, which is why it reaches production.

The fixes, most deployable first

1. Stale-while-revalidate. Keep serving the expired value while one caller refreshes in the background. Nobody waits, the source sees exactly one query, and the cost is data up to one refresh interval staler. Usually the strongest general answer, and it works without coordination between processes.

2. Request coalescing. The first caller to miss acquires an in-process lock for that key and fetches; the others wait for its result. Purely in-application, no infrastructure, low risk. With many processes you get one query per process rather than one globally — usually sufficient.

3. Probabilistic early expiry. Refresh chance rises as an entry approaches expiry, so refreshes spread across time and the herd never forms. Elegant; slightly harder to reason about.

4. Jittered TTLs. Not a fix for a single hot key, but essential for the related failure where many keys populated together expire together. Always jitter.

The deeper question

Why does one key's expiry threaten the database? That is a signal the cache is not an optimisation but a load-bearing dependency.

Two consequences: test with a cold cache — if the system cannot start with an empty one, you have an availability problem that will present after any restart or cache-tier failure, which is precisely when you are already in trouble. And have a fallback: a degraded response or an explicit error beats taking the database down for everyone.

The related failure worth naming

At high read fan-out, invalidations race with reads already in flight, so a cache can be repopulated with a stale value after the invalidation arrives. The robust answer is versioned keys — a version or content hash in the key, so a new version is a new key and the race cannot occur.