Sharding in Practice

Definition

Splitting a dataset across multiple independent stores, each holding a subset, so that capacity and throughput scale horizontally rather than by growing a single machine.

Why it matters

Vertical scaling has a ceiling, and it is reached earlier than expected for write-heavy workloads: a single primary accepts writes at the rate one machine can commit them. Read replicas relieve read load and do nothing for writes.

Sharding is the answer, and it is a one-way door — the shard key propagates into the data model, the access patterns, the application code and the operational tooling, and changing it later is a migration of the entire dataset.

Implementation patterns

Choosing the key is the whole decision. It should: appear in the overwhelming majority of queries, so a request touches one shard; distribute evenly, so no shard is hot; and keep related data together, so common joins stay within a shard.

Typical good keys: tenant or customer identifier for B2B SaaS; user identifier for consumer products; a compound key where a single dimension is too skewed.

Routing by a lookup table, which allows arbitrary placement and rebalancing at the cost of a lookup, or by a hash function, which is stateless and constrains movement.

Cross-shard operations need an explicit answer: scatter-gather for reads, and a saga or compensation for writes, because a transaction cannot span shards.

Failure scenarios

Skew. The chosen key is uneven — one tenant is a hundred times larger than the median, one channel generates more traffic than thousands. The remedy is a compound key adding a dimension along which the hot entity spreads, or dedicated capacity for the outlier.

Cross-shard queries becoming the norm. If most queries need data from several shards, the key is wrong and the system now has the complexity of sharding with none of the benefit.

Resharding unplanned. Shard count changes, and moving data while serving traffic is a bespoke project. Systems that do not plan for it end up with permanently unbalanced shards nobody dares rebalance.

Referential integrity and uniqueness across shards, which the database no longer enforces.

Industry example

Discord partitions messages by channel identifier combined with a time bucket rather than by channel alone. Channel activity is enormously skewed, and a single very busy community channel would concentrate load on one partition permanently. The compound key bounds how large and how hot any partition can become while preserving the access pattern of reading recent messages in a channel.

Shopify shards its platform by merchant, which additionally gives blast-radius isolation: an incident affects one shard's merchants rather than the platform.

Trade-offs

Gains: horizontal write scaling, bounded blast radius per shard, and the ability to place shards by residency or tier.

Costs: no cross-shard transactions, harder queries and reporting, operational complexity multiplied by shard count, a routing layer that must be highly available, and a key decision that is extremely expensive to reverse.

Interview question

Design the sharding strategy for a multi-tenant SaaS product where the largest customer is 200 times the median.

Look for: tenant as the natural key, immediate recognition that skew makes it insufficient alone, and a proposal that isolates the largest tenants onto dedicated shards while hashing the long tail. A strong answer also plans the resharding mechanism from the start and notes that per-tenant sharding conveniently supports residency and isolation requirements.