AWS: Shuffle Sharding

The problem

Standard sharding assigns each customer to one shard. If a customer sends traffic that poisons their shard — a pathological query, a retry storm, a request that triggers a bug — everyone on that shard goes down with them. With eight shards, one bad tenant takes out an eighth of the customer base.

What they did

AWS described shuffle sharding in their Builders' Library. Instead of one shard per customer, each customer is assigned a random subset of workers — say two out of eight.

The combinatorics do the work. There are 28 ways to choose two workers from eight, so two randomly assigned customers share both their workers only about 4% of the time. A customer whose traffic destroys both of their workers takes down only the small number of customers who happened to draw the identical pair; everyone else still has at least one healthy worker and, with a client that retries against its other assignment, continues to be served.

Scale the numbers and it gets dramatically better: 100 workers with 5 each gives combinations in the tens of millions.

The trade-off

It requires clients that can retry against their alternate assignment, or the isolation is theoretical — a customer whose request lands on their degraded worker still fails if they cannot try the other.

It also complicates capacity planning and debugging: "which workers serve this customer" is now a lookup rather than a formula, and per-shard dashboards become less meaningful.

The transferable lesson

Blast radius is a design variable, not a fixed property. Most multi-tenant systems assign tenants to shards by hash without ever asking what happens when one tenant behaves badly — and the answer is usually "everyone in that shard suffers".

The technique applies well beyond AWS's scale: connection pools, worker queues, rate-limit buckets and cache nodes can all be shuffle sharded, and the cost is a mapping table plus retry logic. It is one of the highest-leverage isolation patterns available and one of the least used.