Caching Strategies
Which cache pattern to use, how staleness is bounded, and the failure modes that only appear under load.
Definition
A cache trades correctness for latency and cost: it serves a copy that may be out of date. Every caching decision is therefore a decision about how stale is acceptable, and what happens when the answer is wrong.
The patterns
| Pattern | Mechanism | Fits |
|---|---|---|
| Cache-aside | App checks cache, on miss reads the source and populates | Most read-heavy workloads; the default |
| Read-through | Cache itself loads on miss | Same, with the logic centralised |
| Write-through | Write to cache and source together | Read-after-write consistency needed; slower writes |
| Write-behind | Write to cache, flush to source asynchronously | Very high write rates; risks loss on failure |
| Refresh-ahead | Refresh popular keys before expiry | Predictable hot keys; avoids miss latency |
Invalidation, which is the actual problem
Three approaches, in increasing order of precision and cost:
- TTL only. Simple, bounded staleness, no coordination. The default and usually correct.
- Explicit invalidation on write. Precise, and now the cache is coupled to every writer — including the batch job and the manual data fix that nobody remembers to wire up.
- Versioned keys. Include a version or a content hash in the key so a new version is a new key and the old one expires naturally. Avoids the race between invalidation and in-flight reads, and is frequently the cleanest answer.
Meta's experience with large-scale caching is the reference point for why this is hard: at high read fan-out across regions, invalidations race with reads already in flight, so a cache can be repopulated with a stale value after the invalidation arrives. Preventing that requires the invalidation to be ordered with respect to the write, which is a distributed-systems problem rather than a cache setting.
Failure modes that appear only under load
- Thundering herd / cache stampede. A popular key expires and a thousand concurrent requests all miss and all hit the database at once. Fixes: request coalescing (one caller fetches, the rest wait), a short lock per key, or serving stale while one caller revalidates.
- Synchronised expiry. Many keys populated at the same moment expire at the same moment. Fix: jitter the TTL.
- Cache penetration. Requests for keys that do not exist bypass the cache every time. Fix: cache the negative result briefly, or use a probabilistic filter.
- Hot key. One key receives a disproportionate share of traffic and saturates a single cache node. Fix: replicate that key across nodes, or add a small local in-process cache in front.
- Cache as a hidden dependency. The system cannot survive a cold cache — after a restart or a flush, the origin is overwhelmed. If you have never tested with a cold cache, the cache is not an optimisation, it is a load-bearing component.
Trade-offs
Bought: latency, throughput, and cost relief on the origin. Sold: correctness within the staleness window, a new failure mode on every layer, and debugging difficulty — "it works but shows old data" is far harder to diagnose than an error.
Decide staleness tolerance per data type explicitly: a product price may tolerate seconds, an inventory count may not, a permission check should probably not be cached at all.
Interview question
"A popular cache key expires at peak and the database falls over. Name three different fixes and tell me which you would deploy first."