When Notion sharded its Postgres database in 2021 it created 480 logical shards spread across 32 physical instances, 15 per host. In 2023 it moved to 96 hosts with 5 logical shards each, keeping the same 480. What did the original choice buy, what did it cost, and what would you have had to believe to choose 32 logical shards instead?
Show the full answer Hide the answer
The situation they were in
Notion's block data had outgrown a single Postgres primary, and the decision in front of them was not only "shard" but "how many shards". The naive answer is one logical shard per host, because it is the simplest thing that works and nothing is wasted. The consequence of that answer only appears the next time you need capacity.
What they chose
They decoupled the logical shard count from the physical host count, fixing logical shards at 480 and packing 15 onto each of 32 hosts. 480 was chosen because it divides cleanly by many numbers, so a later move to 40, 48, 96 or 120 hosts is a redistribution of whole shards rather than a re-partitioning of data. In 2023, with some shards near 90% CPU and disk IOPS becoming the constraint, they went to 96 hosts at 5 shards each — the same 480 logical shards, relocated.
Why it fit their constraints
The expensive operation in sharding is not adding a machine; it is changing which shard a row belongs to, because that rewrites the routing rule, invalidates every cached mapping, and requires moving data while serving traffic. Fixing the logical count at a number far above the host count means the routing rule never changes again. A capacity increase becomes a replication-and-cutover exercise per shard, which is a known, rehearsed operation with a rollback.
In trade-off language: they paid a fixed, known cost up front to convert a future re-partitioning into a future rebalance. That is the general shape of buying reversibility.
What it cost them
- Per-shard overhead multiplied by 480, not by 32: connections, vacuum, monitoring, backup jobs, and 480 of every schema migration.
- Cross-shard queries are harder to avoid with more shards, so application discipline about the partition key has to be enforced earlier.
- More moving parts to observe. A single slow shard is 1/480th of traffic and can hide behind aggregate dashboards, which is exactly the diagnosis problem the 2023 work ran into.
What you would have to believe to choose 32
That you will never need more than 32 hosts, or that re-partitioning later will be affordable. The first requires a growth forecast nobody has; the second is the claim worth attacking, because re-partitioning under live traffic is the work teams describe as the hardest thing they have done. Choose the smaller number only when the data has a natural bound — one shard per country when you operate in a fixed set of countries, one per regulated entity — because then the count is set by the world rather than by a guess.
When this is the wrong answer
If a single Postgres primary is not yet the constraint, none of this applies and doing it anyway costs you a year. Vertical scale, read replicas and removing your worst query buy time measured in years for a fraction of the effort, and the migration you avoid is the one that goes wrong. Figma's published account of delaying horizontal sharding by partitioning vertically first is the same judgement: sharding is what you do when writes or working set exceed one machine, not when reads are slow.
What a strong answer adds
The senior layer is naming which future decisions the choice keeps open, and at what price — and noticing that the price is paid immediately while the benefit arrives in an unknown year. That is the same calculation as buying an option, and the discipline is to say what the option is worth: here, avoiding one re-partitioning of a live dataset. If the answer is "we have no idea whether we will ever exercise this", the cheap version is to write down how you would re-partition and accept the risk.