Cache Key Completeness

A cache maps a key to a response. The cache is correct only if every input that changes the response is part of the key. Anything varying the response and missing from the key means one request's response is served to a different request that should have received something else.

Most caching discussion concerns staleness — serving old data. Key incompleteness is a different and more severe failure: serving someone else's data. It is not a performance bug; it is a security incident.

Why it matters

The inputs that vary a response are more numerous than they first appear: session identity, permissions, locale, currency, device class, feature flags, experiment assignment, content-visibility rules, and compression negotiation. Any of these missing from the key produces incorrect responses, and the incorrect case is often the rare one — so it survives testing and appears in production intermittently.

Implementation patterns

• Cacheability is opt-in, never opt-out. Responses are uncacheable by default and an endpoint declares otherwise deliberately. Then the failure mode of forgetting is a missed optimisation rather than a leak. A default-cache rule with an exception list guarantees that any endpoint the list misses is cached wrongly.
• Split the cacheable shell from the personalised fragment. The article, the thread, the product page is identical for everyone; the vote state, the cart count, the personalised banner is small and per-user. Cache the shell and compose the fragment at the edge. This is what makes personalised pages cacheable at all, and it is the highest-leverage move available.
• An explicit, reviewed list of varying dimensions in the key. If the list is long, the response is not really cacheable and should be split instead.
• Avoid Vary on high-cardinality headers. Vary: Cookie is technically correct and practically useless — every user has a distinct cookie, so the cache stores a copy per user, the hit rate approaches zero, and a correctness bug becomes a memory problem.
• A fitness function asserting that no authenticated response is cached under a key lacking user identity. This is the only control that reliably prevents recurrence, because the defect is reintroduced by ordinary changes rather than by carelessness.

Industry example

A discussion platform puts a caching reverse proxy in front of its application. Anonymous traffic gets much faster; logged-in users see no improvement and occasionally see another user's content.

Those two findings are the same bug. Logged-in responses vary by user, the key is derived from the URL, so personalised responses are either excluded from caching — hence no improvement — or cached incorrectly. Wherever the exclusion rule is imperfect (one endpoint, one header, one code path), the incorrect case leaks through. That is the "occasionally".

The resolution that actually works is architectural rather than configurational: make the thread and comment tree a cacheable shared object, keep vote state and subscription status as a small per-user fetch, and compose them. The platform then gets caching for the expensive part and correctness for the personal part.

The same failure appears at CDN scale, where an incomplete key during a viral event either serves the wrong variant to millions of people or fragments the cache so badly that nothing is hot.

Failure scenarios

• Session-varying responses cached under a URL-only key — the canonical data leak.
• Compression or format negotiation omitted, serving a compressed body to a client that cannot decode it.
• Locale or currency omitted, showing prices in the wrong currency to a fraction of users.
• A new header introduced by a feature, varying the response without being added to the key.
• Caching added at the infrastructure layer without the application declaring what varies, so the proxy cannot possibly know.

Trade-offs

A complete key is more specific, which means a lower hit rate — every added dimension multiplies the number of cached variants. That tension is real, and the wrong resolution is to drop a dimension to improve the hit rate.

The right resolution is to restructure the response so that the highly-varying part is small and fetched separately, leaving a large, low-variance, highly cacheable object. When you find yourself wanting to remove a dimension from a key, that is the signal to split the response instead.

Interview question

"You add a CDN and hit rate is 30%. A colleague suggests removing Vary: Accept-Language to improve it. What do you say, and what would you do instead?"