Hot Key

Partitioning distributes keys evenly. Traffic is distributed by users, and users do not behave evenly: a viral video, a celebrity account, a flash-sale item, a trending thread or one enormous tenant concentrates a large fraction of requests onto one key. No hashing scheme fixes this, because the imbalance is in the workload, not in the mapping.

The typical shape is severe — a small percentage of keys accounting for the large majority of requests — and it is usually sudden, which is what makes it an incident rather than a capacity plan.

Why it matters

A hot key defeats horizontal scaling. Adding nodes gives you more total capacity and does nothing for the one node serving the hot key, so the system's effective limit is set by a single machine while the fleet sits idle. Teams frequently add capacity, see no improvement, and conclude something deeper is wrong.

Implementation patterns

• Detect first. Per-key or per-partition request counters with a top-N sketch. Aggregate metrics hide hot keys by construction — mean utilisation looks fine while one node saturates.
• Read replication. Replicate the hot key to many nodes and spread reads. Effective for read-heavy hot keys, which is most of them.
• Key salting. Split the key into key#0 … key#N sub-keys written round-robin and read as a scatter- gather. Effective for hot writes, such as a counter on a viral item, at the cost of read complexity.
• Request coalescing. Concurrent requests for the same missing key are collapsed into one upstream fetch, with the rest waiting on its result. Turns a thundering herd into a single request.
• Local caching in front of the shard, absorbing the hottest keys before routing.
• Bounded-load consistent hashing, so an overloaded node overflows to its neighbour rather than falling over.
• Dedicated handling. Above a threshold, route the hot key to a purpose-built path — a pinned in-memory cache, a pre-rendered artefact, a static object at the edge.

Industry example

A video platform experiences a viral event: one video receives millions of requests within minutes. Everything about the steady-state design — popularity-tiered storage, regional caches, an origin sized for aggregate load — is calibrated for a distribution this event violates.

The controls that matter are all about collapsing duplicate work:

• Origin shielding, so the thousands of edge nodes that all miss simultaneously go through a small set of parent caches rather than all reaching origin.
• Request collapsing at every cache layer, so N concurrent misses for one object produce one upstream fetch.
• Stale-while-revalidate, so the expiry of a hot object does not produce a synchronised stampede.
• Cache warming where the event is predictable — a scheduled premiere, a product launch — pushing the object to the edge before demand arrives.

The general principle: for a hot key, the goal is not to serve the load faster but to ensure the work is done once and everyone else reads the result.

Failure scenarios

• Detection only in aggregate, so the hot key is invisible until a node fails.
• Scaling out and expecting improvement.
• Cache expiry synchronising a stampede — every replica's copy of the hot object expires at the same instant.
• Salting writes without adjusting reads, producing counts that are silently wrong.
• A hot key that is also a hot lock, where the contention is on a mutex or row lock rather than on throughput, and replication does not help at all.

Trade-offs

Every mitigation costs something. Replication costs storage and consistency (replicas diverge briefly). Salting costs read complexity and makes strongly consistent aggregates harder. Coalescing costs latency for the waiting requests. Dedicated paths cost a second code path that is exercised rarely and therefore rots.

The judgement is whether hot keys are a routine property of the workload — in which case build for them — or a rare event, in which case detection plus a manual playbook may be the better investment.

Interview question

"Your traffic is 1% of keys accounting for 90% of requests. Walk me through how consistent hashing, replication, request coalescing, salting and hot-key detection combine — and tell me which of them helps a hot write rather than a hot read."