Cache-Loss Survivability

A cache serving the overwhelming majority of reads is not an optimisation. It is load-bearing capacity, and the database behind it has never been sized for the traffic it hides.

When the cache fails, the origin receives many times its designed load, saturates, and its slow queries hold connections — so even the fraction it could have served fails too. This is a capacity event, not a correctness event, and the database cannot survive it by design.

Why it matters

It is one of the most common total outages in consumer platforms, and it happens at the worst possible moment, because cache pressure and traffic peaks coincide. It is also entirely preventable with controls that cost little.

Implementation patterns

Ordered by effectiveness:

• Request coalescing (single-flight). A thousand simultaneous misses on one key produce one origin query and the rest wait for its result. Highest leverage available, and it also fixes the ordinary expiry stampede rather than only the total-failure case.
• A hard concurrency limit on origin access, bounding in-flight queries regardless of arrival rate. Excess is rejected or degraded — queueing is what turns saturation into collapse.
• A small in-process second tier per instance with a short TTL, which survives the shared cache's failure and absorbs the hottest keys. Under a skewed workload that is a large share of traffic.
• Serve stale on origin error, which requires retaining expired entries rather than evicting them.
• Priority-based shedding: core pages and checkout survive, recommendations and personalisation go first.

Invalidation choices that shrink the blast radius:

• Prefer TTL plus stale-while-revalidate over precise invalidation wherever the data tolerates it. Precise invalidation is correct and fragile; a short TTL is approximate and robust.
• Version keys rather than deleting them, so there is never a window where the key is absent and everything stampedes.
• Never make invalidation a synchronous dependency of a write, which would couple durability to a cache's availability.

Industry example

Retail platforms such as Nykaa and Myntra face this precisely during a sale, when cache dependence is highest and the origin's headroom lowest. The controls above are cheap; the rehearsal is what teams skip.

Turn the cache off in a controlled test and watch what happens. Almost every team that has not done this discovers their database cannot survive it — and discovering that during a sale is the expensive way to learn.

Failure scenarios

• Cache node loss becoming an origin outage, the canonical case.
• Synchronised TTL expiry on a popular key, producing a stampede without any cache failure at all.
• Invalidation on write coupling durability to the cache's availability.
• No local tier, so a shared cache failure is total rather than partial.
• Untested failure path, so the controls exist in code and have never run under load.

Trade-offs

Every control here costs something. Coalescing adds latency for the waiters and a coordination mechanism. Concurrency limits reject requests the origin might have served. A local tier adds a second source of staleness, so a user can see different values on consecutive requests hitting different instances.

The judgement is that bounded degradation beats unbounded failure, and the staleness introduced by a local tier is measured in seconds while the alternative is measured in outage minutes. The only control with no downside is coalescing, which should be the default in any cache client.

Interview question

"Your cache cluster is lost at the peak of a sale. Take me through the next sixty seconds in your current architecture, then tell me what you would have built differently — and which single change gives the most protection per hour of work."