Invariant-Aligned Partitioning

Strong consistency across a distributed system is expensive. Strong consistency within one partition owned by one writer is nearly free — it is just ordinary local ordering.

The principle: choose the partition key so that each invariant that must hold exactly is contained within a single partition. Then the system is strongly consistent where it must be and eventually consistent everywhere else, without any global coordination.

Why it matters

It converts the hardest problem in distributed systems into a data-modelling decision made once. It is also among the most expensive decisions to change later, because re-partitioning a live system with ordering guarantees means keys landing in two places during the transition — which breaks the guarantee at exactly the worst moment.

Implementation patterns

• Enumerate the invariants first, then derive the key. "A balance may not go negative" is per account, so the key is the account. "An order may match at most once" is per instrument, so the key is the instrument. "Stock may not be oversold" is per location-SKU, so that pair is the key.
• Route all writes for a key through one owner — a shard, a partition, a single-threaded actor. The serialisation is then local and needs no consensus.
• Accept eventual consistency across keys, which is almost always what the business actually requires.
• Over-provision partitions. Increasing partition count later rehashes keys and breaks per-key ordering during the transition. Extra partitions cost overhead; too few cost a migration.
• Handle the invariant that spans partitions explicitly — a transfer between two accounts — with a saga, a two-phase protocol, or by making one of the two the owner of the operation. There is no free version of this and pretending otherwise is where correctness bugs live.

Industry example

A crypto exchange such as CoinDCX has two different invariants with two different natural keys. Wallet operations are per account: a stale balance permits spending money that is not there, and the counterparty has already been paid. Order matching is per instrument: two matches against the same resting order is not a display bug, it is two trades.

So the system is partitioned two ways and the boundary between them is where the design is difficult — placing an order must check a balance in one partition and enter a book in another, which is precisely the cross-partition invariant that needs an explicit protocol.

Trading platforms, wallets, ledgers and inventory systems all reduce to this same shape.

Failure scenarios

• A key chosen for load distribution rather than for the invariant, which distributes beautifully and provides no consistency where it is needed.
• A hot key — one enormous account or one heavily traded instrument — that exceeds a single partition's capacity and cannot be split without losing the guarantee.
• Cross-partition invariants discovered late, when the transfer feature is designed.
• Re-partitioning a live system, breaking ordering during the transition.
• Consumers that read from a partition and process concurrently, discarding the ordering the partitioning paid for.

Trade-offs

Aligning partitions to invariants can conflict with aligning them to load. If one instrument carries most of the volume, invariant-aligned partitioning gives you a hot partition and no way to split it. The available responses are vertical scaling of that partition, a dedicated deployment for it, or a redesign of the invariant — and all three are worse than the problem being avoided, which is why the hot-key case should be anticipated rather than discovered.

The alternative — partition for load and coordinate for consistency — buys smooth distribution at the cost of a consensus protocol on the write path, with its latency and its availability characteristics inherited by every operation.

Interview question

"Your exchange partitions wallets by account and order books by instrument. A customer places an order, which touches both. Walk me through the protocol, and tell me what happens when the second partition is unavailable after the first has committed."