Shard Key Selection
Choosing the attribute that determines a record's partition, which fixes the system's distribution, query patterns and future flexibility.
The most consequential and least reversible decision in a partitioned data store. Changing it later means rewriting and redistributing everything.
Three criteria, and a good key satisfies all three:
Even distribution. A key with skewed values produces hot partitions — a large tenant, a popular product, a timestamp when writes are chronological. Adding capacity does not help, because the load is on one partition.
Query alignment. Queries that include the shard key are routed to one partition; queries that do not must fan out to all of them, which is slow and scales badly. The key should reflect the dominant access pattern, and that requires knowing the access patterns before choosing — which is why data modelling precedes technology selection here.
Transaction locality. Operations that must be atomic should fall within one partition, since cross-partition transactions are expensive or unavailable.
Common strategies: hash for even distribution with no range queries; range for range scans with a hotspot risk; composite keys combining a high-cardinality prefix with a sort component, which is usually the practical answer; and directory-based lookup for flexibility at the cost of an extra hop and a shared dependency.
Plan resharding from the beginning — consistent hashing or a virtual-node scheme limits how much data moves when the partition count changes.