Write Contention

Throughput problems are usually solved by adding capacity, because most work is independent. Write contention is different: every writer needs the same row, so the work is dependent on one item of state and must serialise. The system's ceiling is the rate at which a single row can be updated — typically a few thousand operations per second — regardless of how many machines exist.

Adding capacity makes it worse, because more application instances means more concurrent attempts at the same lock, and time spent waiting rises rather than throughput.

Why conventional techniques do not apply

• Caching helps reads. The decrement cannot be cached, and a cached availability read used to make the reservation decision is how overselling happens — it works under normal load and fails during exactly the surge that made caching seem necessary.
• Read replicas are irrelevant to a write problem.
• Autoscaling responds in minutes to an event arriving in seconds, and adds contenders.
• Sharding does not help when one logical entity receives all the traffic.

Implementation patterns

• Admission control before the contention point. If 500 units exist, admitting 5,000 requests and rejecting 4,500 after they have consumed database resources is waste. Rejecting at the edge costs microseconds. This is the single most important control.
• Segment the counter. Divide the quantity into independent segments with their own rows, routing buyers across them, so contention divides by the segment count. Costs complexity when one segment empties while others have stock, and works because approximate fairness is acceptable.
• Atomic operations in an in-memory store, with the outcome persisted asynchronously — removing disk and transaction overhead from the contention point, at the price of a careful durability design.
• Queue the intent and process serially, which converts contention into an ordered queue with fair first-come-first-served semantics and a controllable rate.
• Pre-allocate entitlements — a lottery or registration window — so the moment of sale is a redemption rather than a race.
• Conditional atomic update rather than read-then-write, so there is no window between check and act.

Industry example

Extreme synchronised-demand commerce events are the archetype: millions of customers attempting to buy a small number of items at a scheduled instant. Everything the platform normally relies on for scale is inapplicable, and the honest ceiling is a property of one row.

The design insight that generalises is about which invariant is being protected. Inventory correctness is the one effect that cannot be compensated — a double charge is refundable, a double sale of a limited item is not. So the reservation path is protected with strict consistency and everything around it is shed: recommendations, reviews, search facets, view tracking.

Identify the invariant that cannot be compensated, protect it properly, and refuse to pay for strong consistency anywhere else.

Failure scenarios

• Adding capacity as the response, which increases contention.
• Optimistic concurrency under high contention, where retries multiply exactly when the resource is busiest.
• Holding the lock across a network call — a payment provider, a rules service — serialising the hottest resource behind the slowest participant.
• Segmenting writes without adjusting reads, producing counts that are silently wrong.
• Using a cached read for the reservation decision.

Trade-offs

Every mitigation costs something. Segmentation costs exact fairness and adds complexity at the boundaries. In-memory decrements cost durability guarantees. Queueing costs immediacy and requires a user experience that communicates waiting honestly. Pre-allocation changes the product.

The judgement is whether the contention is a routine property of the workload — in which case build for it — or a rare scheduled event, in which case admission control plus a queue may be sufficient without re-architecting the data model.

Interview question

"A million users try to buy 500 units at the same second. Walk me through where the bottleneck is, why adding servers makes it worse, and four things you would do — in the order you would do them."