A popular cache key expires at peak. A thousand concurrent requests miss and hit the database simultaneously, and it falls over. Name three fixes and pick one to deploy first.

What is being tested

Recognition of the cache stampede, and whether you can pick the fix that is deployable during an incident rather than the most elegant one.

The mechanism

Cache-aside logic is: check the cache, on miss read the source and populate. That logic contains an unstated assumption — that only one caller will miss at a time. At peak, when a hot key expires, a thousand callers miss within the same few milliseconds and every one of them independently queries the source. The database receives a thousand copies of an expensive query at once.

The database was comfortably serving the 0.1% of traffic that normally missed. It cannot serve 100%.

The fixes

1. Request coalescing (single-flight). When a miss occurs, the first caller acquires an in-process lock for that key and fetches; the others wait for its result. Reduces a thousand queries to one per process. Simple, purely in-application, and needs no infrastructure. With many processes you get one query per process rather than one globally, which is usually sufficient.

2. Stale-while-revalidate. Keep serving the expired value while one caller refreshes in the background. Nobody waits, the source sees exactly one query, and the cost is that users see data up to one refresh interval staler. For most cacheable data this is entirely acceptable and it is the strongest general answer.

3. Probabilistic early expiry. Each read, with a small probability that increases as the entry approaches expiry, triggers a refresh. Refreshes spread naturally across time and the herd never forms. Elegant, no locks, slightly harder to reason about.

4. Jittered TTLs. Not a fix for a single hot key, but essential for the related failure where many keys populated together expire together. Always jitter.

What to deploy first

Stale-while-revalidate, if the data tolerates it, because it works without coordination between processes and removes the latency cliff as well as the load spike. If staleness is unacceptable for this key, deploy request coalescing, which is a small, local, low-risk change.

Longer term, add jitter everywhere and consider refresh-ahead for the small set of keys known to be hot.

The deeper question

Why does a single key's expiry threaten the database? That is a signal the cache is not an optimisation but a load-bearing dependency. Two things follow:

• Test with a cold cache. If the system cannot start with an empty cache, you have an availability problem that will present itself after any restart, flush, or cache-tier failure — precisely when you are already in trouble.
• Have a fallback. Serving a degraded response, a default, or an explicit error is better than taking the database down for everyone.

The related failure worth naming

At high read fan-out across regions, invalidations race with reads already in flight, so a cache can be repopulated with a stale value after the invalidation arrives. This is the problem behind the observation that cache invalidation is genuinely hard rather than merely fiddly, and the usual robust answer is versioned keys — put a version or content hash in the key so a new version is a new key and the race cannot occur.