advanced
2 min answer
A ride-hailing platform maintains a materialised view of driver availability by geographic cell. What are the design decisions, and what breaks?
Show the full answer Hide the answer
The design decisions
- Update mechanism: incremental from a change stream, or periodic recomputation. Incremental is far cheaper and accumulates drift; periodic is expensive and self-correcting. Most systems need both — an incremental path for freshness and a periodic recomputation that corrects drift.
- Freshness target, which is a product decision. Driver availability three seconds stale is acceptable for ranking candidates; it is not acceptable for the commit that assigns one.
- Consistency with the source, which is eventual by construction. The view is a projection and the source is authoritative, and any code path that treats the view as authoritative is a bug waiting for a race.
- Rebuild capability, which must exist and must be fast enough to be usable. A projection that has never been rebuilt cannot be trusted to be rebuildable when it is wrong, and it will be wrong.
What breaks
- Drift. Incremental updates miss events, apply them twice, or apply them out of order, and the view diverges silently. Nothing errors — the numbers are simply wrong — which is why a periodic recomputation and a divergence metric are required rather than optional.
- The view used for a decision that needs the source. Ranking from a stale view is correct; committing an assignment from it is not. The design must make that separation explicit: approximate for selection, exact for commitment via a conditional write against the authoritative record.
- Rebuild taking longer than the tolerance. If recomputing takes six hours and the view is on the critical path, a bug is a six-hour outage.
- Boundary artefacts. Geographic cells produce edge effects — a driver just outside a cell boundary is invisible to it — which is a modelling problem the view's structure creates and which must be handled by querying neighbouring cells.
The pattern that makes it safe
Approximate from the view, exact at the point of commitment. Rank candidates from the stale projection — cheap, fast, good enough — then commit with a conditional write against the authoritative record that succeeds only if the resource is still free. If it fails, advance to the next candidate rather than retrying, since the resource was taken.
That is optimistic concurrency, and it works because conflicts are rare. It stops working as contention rises — during a shortage — at which point the answer is to reduce contention by partitioning rather than to introduce locking.