A multi-tenant database is being sharded. One tenant holds 40% of the data and generates 60% of the queries. What is your shard key and what do you do about that tenant?

What is being tested

Whether you recognise that the skew is the design problem and the shard key is only half the answer. A candidate who picks a key and moves on has not engaged with the question.

The key

Shard by tenant_id. Almost every query in a multi-tenant system filters by tenant, so this key routes nearly all queries to a single shard, keeps a tenant's data together for joins, and gives you isolation, per-tenant backup and restore, and data residency placement for free. Hashing on a row-level ID would distribute evenly and make every query a scatter-gather, which is the classic mistake.

Now the actual problem

tenant_id distributes work by tenant size, and tenant sizes follow a power law. One tenant at 40% of the data means one shard carries 40% of the data and 60% of the load while others idle. Adding shards does not help, because that tenant's data cannot be split by the key you chose.

Three responses, usually combined:

1. A dedicated shard for the large tenant. The simplest and most effective. Give them their own instance, sized for their load. Isolation cuts both ways and both are valuable: their traffic cannot hurt anyone, and nobody can hurt them. This is also frequently a commercial win — dedicated infrastructure is a legitimate enterprise product tier.

2. Sub-sharding within that tenant. For a tenant genuinely too large for one instance, introduce a composite key: tenant_id plus a secondary dimension that queries already filter on — region, workspace, project. This preserves single-shard routing for most queries while splitting the load. The secondary dimension must come from the access pattern, not be invented.

3. A directory rather than a hash. A lookup table mapping tenant to shard makes placement a decision instead of a calculation, so tenants can be moved as they grow. This is the right structure for multi-tenant systems, because tenant sizes change and hashing gives you no way to respond. The cost is a directory service on the critical path — cached aggressively, since it changes rarely.

The alternative worth naming

Pinterest's approach embeds the shard location inside the identifier itself, so any object's home shard is derivable from its ID with no lookup and no directory to fail. That trades rebalancing flexibility for radical operational simplicity, and for a read-heavy workload with uniform objects it is an excellent trade.

For multi-tenant SaaS with power-law tenants it is the wrong one, because the ability to move a tenant is exactly the capability you need most. Knowing why the same technique is right in one context and wrong in another is the point.

Operational planning

• Plan the rebalancing mechanism before you need it. You will need it during a crisis, and building live tenant migration under pressure is how outages get extended.
• Monitor per-shard load, not aggregate load. Averages hide exactly this problem — the mean shard looks fine while one is on fire.
• Set a growth threshold that triggers a move automatically, rather than waiting for saturation.
• Cross-shard uniqueness for user-visible identifiers needs its own mechanism; two shards will otherwise allocate the same one.
• Schema migrations are now N migrations, and they will be in different states. Every migration must be compatible with both.

Before sharding at all

Sharding is the last resort of database scaling, and it is the only step that changes the application's model of the world — no joins across shards, no cross-shard transactions, no simple migrations, no cheap change of mind. Exhaust indexing, query optimisation, caching, and read replicas first. Then choose the key as though you cannot change it, because in practice you cannot.