A product catalogue page does 40,000 reads per second against a database that can serve 5,000. Walk me through the caching design, including what happens at 3 AM when the cache is empty.

What the interviewer is testing

Whether you can design a cache including its failure modes, rather than saying "put Redis in front of it".

The base design

Cache-aside with Redis: on read, check the cache; on miss, read the database, populate the cache with a TTL, return. Writes invalidate the key.

At a 95% hit rate, 40,000 reads per second become 2,000 database reads — comfortably inside capacity. That is the easy part. The design lives in the remaining questions.

The 3 AM question: cold cache

An empty cache means all 40,000 requests per second go to a database that can serve 5,000. The database saturates, latency climbs, requests time out, retries arrive, and the cache never fills because nothing completes. The system cannot recover on its own — this is why cache-dependent systems fail to restart under load.

Three defences, and a serious answer names at least two:

• Request coalescing (single-flight): concurrent misses for the same key result in exactly one database read; the rest wait for it. This alone converts 40,000 concurrent misses into one read per distinct key.
• Cache warming on startup for the known-hot set, before the instance is added to the load balancer.
• Load shedding during recovery: serve a reduced catalogue or a static fallback to a fraction of traffic until the hit rate recovers.

Stampede on expiry

Even warm, a popular key expiring causes the same problem at smaller scale. Fixes: probabilistic early expiry (refresh a key slightly before its TTL, with the probability rising as expiry approaches, so refreshes spread out) and jittered TTLs so keys written together do not expire together.

Consistency

Cache-aside has a known race: reader misses, reads the database, and before it writes to the cache a writer updates the database and invalidates. The reader then writes its stale value, which persists for a full TTL. Mitigations are a short TTL as a backstop, versioned keys, or delete-on- write followed by a second delayed delete.

Failure of the cache itself

If Redis becomes unavailable and every request falls through, you have the cold-cache scenario without warning. The cache is now a critical dependency, so it needs its own availability design: a replica, a circuit breaker in front of it, and a load-shedding policy for when it is gone.

What a strong answer adds

Noting that a catalogue is a strong CDN candidate — much of this traffic should not reach the origin at all — and that per-key TTLs by volatility (price versus description) beat one global TTL.