A retail platform's cache layer becomes unavailable during a high-traffic sale and all traffic lands on the primary database. What design prevents a cache outage from becoming a database outage?

Why this is a classic total outage

Under a sale, the cache may be absorbing the overwhelming majority of reads. If it disappears, the database receives many times the traffic it has ever been sized for, saturates, and the resulting slow queries hold connections — so even the traffic it could have served fails. The cache failing is a capacity event, not a correctness event, and it is one the database cannot survive by design.

The controls, in order of effectiveness

• Request coalescing (single-flight). When a thousand requests miss on the same key, one goes to the database and the rest wait for its answer. This is the single highest-leverage control and it also fixes the ordinary stampede on expiry, not just the total-failure case.
• A concurrency limit on database access, so the number of in-flight queries is bounded regardless of incoming request rate. Excess requests are rejected or served degraded rather than queued — queueing is what turns saturation into collapse.
• A local in-process cache as a second tier. Small, per-instance, short TTL. It survives the shared cache's failure and absorbs the hottest keys, which under a skewed retail workload is a large share of traffic.
• Serve stale on error. If the cache is up but the origin is failing, returning expired content is far better than an error page for a product listing. This requires keeping stale entries rather than evicting on expiry.
• Load shedding by priority. Product pages and checkout survive; recommendations, "customers also bought", and personalisation are shed first.

The invalidation design that reduces the blast radius

• Prefer TTL plus stale-while-revalidate over precise invalidation wherever the data tolerates it. Precise invalidation is correct and fragile; a short TTL is approximate and robust, and it degrades gracefully.
• Version keys rather than deleting them. Writing a new key with a new version and letting the old expire avoids the window where a key is missing and every request stampedes.
• Never let invalidation be a synchronous dependency of a write. A write that fails because a cache node is unreachable has coupled durability to a cache's availability.

The rehearsal

Turn the cache off in a controlled test and watch what happens. Almost every team that has not done this discovers their database cannot survive it, and discovering that during a sale is the expensive way to learn.