Amazon Dynamo: Always Writeable

The problem

For Amazon's shopping cart, the commercially unacceptable failure is a rejected write. A customer who cannot add an item is a lost sale, and the cost of that is immediate and measurable. A customer who briefly sees a slightly stale cart is not.

That asymmetry inverts the usual database default, which prioritises consistency and rejects writes when it cannot guarantee it.

What they did

The 2007 Dynamo paper describes a system designed around "always writeable". The mechanisms have since become standard vocabulary:

Consistent hashing with virtual nodes so that adding or removing a node redistributes a small fraction of keys rather than remapping everything. Sloppy quorum with hinted handoff so that a write succeeds even when the intended replicas are unreachable, with the data handed to a temporary holder and forwarded later. Vector clocks to record causality, so the system can tell a genuine concurrent write from a stale one. Merkle trees for anti-entropy between replicas.

The critical design decision is the last one: Dynamo does not resolve conflicts. It surfaces divergent versions to the application, and for the cart the resolution is a union of the two versions — which is why deleted items occasionally reappeared in Amazon carts. That was a deliberate, priced trade: resurrecting a removed item is a mild annoyance, and losing an added item is revenue.

The trade-off

Application-level conflict resolution is a genuine burden. Every consumer of the datastore must understand versioning and implement a merge, and most teams do not want to. This is precisely why later managed systems in this lineage default to last-write-wins — which is easier and silently discards data.

The transferable lesson

Choose the consistency model from the cost asymmetry of the two failure modes, not from a general preference. Ask which is worse in this specific domain: rejecting a valid operation, or accepting two conflicting ones. For a cart, a "like", a view counter or a draft document, availability wins. For inventory allocation, a balance or a seat booking, it does not.

And if you choose availability, decide the merge semantics deliberately at design time — because if you do not, the framework has chosen last-write-wins on your behalf.