Integrated Cache Tier

When caching is left to callers, every team that reads a datastore eventually builds a cache in front of it. The organisation ends up with many independent implementations of invalidation — the hardest correctness problem in the stack — each with its own TTL guess, its own staleness bug, and its own undefined behaviour when the cache is unavailable.

An integrated cache tier moves the cache behind the storage client's interface. Callers issue reads; the tier decides whether they are served from cache or origin, and invalidation is driven by the store's own replication stream rather than by application code remembering to invalidate.

Why it matters

Invalidation driven by the change stream is correct by construction in a way application-driven invalidation never is. The application path has a window between commit and invalidate, has code paths that forget, and has no way to catch writes made by other services or by a migration script. The replication log sees every write, in order, including the ones nobody remembered.

The second argument is failure behaviour. When each team caches independently, the system's behaviour when Redis is down is the union of N different guesses. Centrally, it is a policy.

Implementation patterns

• Invalidate from the replication/change stream, so invalidation follows the commit rather than racing it.
• Delete rather than update unless the new value is complete and cheap — a partial value written into cache is a corruption that persists until TTL.
• Keep a TTL as a bound on missed invalidations, not as the primary mechanism.
• Per-use-case consistency: write-through or a short bypass marker for keys needing read-your-writes, ordinary async invalidation elsewhere. Applying the strict mode everywhere destroys the hit rate.
• Negative caching for misses, with a shorter TTL, so a repeatedly-requested absent key does not hit the origin every time.
• Request coalescing so simultaneous misses on one key produce a single origin read.
• Topology awareness: on a shard failover or split, invalidate the affected key range, because the cache's contents are now of uncertain provenance.
• An explicit cache-down policy — either the origin is provisioned for full uncached load, or the tier sheds load when the cache is unavailable.

Industry example

Uber's CacheFront sits in front of Docstore, its sharded MySQL-backed document store. The workload is the ideal one for caching — reads concentrated on a small set of recently-active entities, so the working set is small and ages out naturally. The design point worth taking is that it is part of the storage tier rather than a per-team concern, so invalidation is driven by the change stream and every consumer inherits the same semantics without implementing anything.

Failure scenarios

• Application-driven invalidation that misses writes from migrations, backfills or other services.
• Writing partial objects into the cache, poisoning it until expiry.
• Cold cache after failover, producing a thundering herd onto a database at its most fragile moment.
• The origin no longer sized for uncached load, so losing the cache takes down the database — at which point the cache was an undeclared primary datastore.
• Read-your-writes applied globally, collapsing the hit rate and eliminating the benefit.
• Cache and database in the same failure domain, so both are lost together.
• Unbounded key growth with no eviction policy, turning a cache into a memory-bound store that evicts unpredictably under pressure.

Trade-offs

Integration means the storage team owns a cache and its failure modes, becoming responsible for a component that was previously somebody else's problem — a real organisational cost, and the reason it is usually only worth doing at a scale where many teams share the store.

It also removes per-caller control. A team that genuinely needs different consistency or a different TTL now negotiates with a platform rather than editing its own code, which is slower and is the point: uniformity is what is being bought.

The trade is per-team flexibility and platform-team ownership burden in exchange for one correct invalidation implementation, uniform semantics and controlled failure behaviour. Below a certain organisational size the flexibility is worth more; above it, the N-implementations problem dominates decisively.

Interview question

"Reads on our document store are 95% concentrated on entities touched in the last hour. Design the cache — and then tell me what happens on the first request after a shard fails over, and what happens to the database if the cache cluster is lost entirely at peak."