A food-delivery platform must match orders to nearby couriers while supply and demand change every few seconds. Which information needs strong consistency, which can be stale, and how should dispatch handle a courier who accepted a job while their availability record was out of date?
Show the full answer Hide the answer
The consistency split
- Courier location: deliberately stale, and that is correct. Locations arrive every few seconds from hundreds of thousands of devices. Trying to make them strongly consistent would cost more than the dispatch decision is worth, and a location three seconds old is good enough to rank candidates.
- Courier availability for ranking: stale is fine. The candidate set is a heuristic.
- The assignment itself: strongly consistent, single-writer per courier. A courier assigned to two orders is a physical failure, a customer complaint and a refund. This is the one write that must not race.
- Order state machine: strongly consistent per order. Transitions must be ordered and idempotent.
Handling the stale acceptance
The correct design does not try to prevent the race; it resolves it atomically at the point of commitment:
- Rank candidates from stale data — fast, approximate, cheap.
- Offer to the top candidate.
- On acceptance, attempt a conditional claim on the courier: assign only if currently unassigned, as a compare-and-set against the courier's authoritative record.
- If the claim fails, the courier was taken; return immediately to the next candidate rather than retrying.
This is optimistic concurrency, and the reason it fits is that conflicts are rare. Most offers do not race. Pessimistically locking every candidate during ranking would serialise dispatch across the city for a conflict rate of a few percent.
The second-order problem
Offer expiry. A courier who does not respond holds a soft reservation. Too short and you churn through candidates; too long and the order sits while couriers idle. The expiry must be a tuned parameter with its own metric, and it must be enforced by the system rather than trusted to the client — a phone that lost connectivity cannot release its own reservation.
The failure mode that defines the domain
Dispatch decisions cause physical movement, and physical movement cannot be rolled back. A courier dispatched to the wrong restaurant is a real cost. This is why the assignment is the strongly consistent operation while everything feeding it is not: you can be approximate about who to ask and must be exact about who was told.