advanced 2 min answer

A multi-region active-active deployment allows writes in every region. Classify entities by the conflict strategy each needs, and describe how cross-region failover should be tested.

multi-regionactive-activecrdthome-regionconflict-resolution
Show the full answer Hide the answer

The classification that makes this tractable

Active-active is not a system property; it is a per-entity decision. Applying one strategy globally is the error that produces either unacceptable latency or silent data loss.

Class 1 — last-writer-wins is safe. Entities where the latest value is correct and losing an intermediate write is harmless: a user's display preferences, a profile field, a "last seen" timestamp, cached derived values. The test is whether a lost concurrent update causes any harm. Usually the honest answer is no, and this class is larger than people expect.

Class 2 — conflict-free data types. Entities that are naturally mergeable: counters (grow-only or positive-negative), sets with add/remove semantics, collaborative documents, shopping carts. Concurrent operations from different regions merge to the same result without coordination, at the cost of metadata growth and, for some types, no way to express "remove and never re-add."

Class 3 — home region required. Entities with an invariant that must hold globally: account balances, inventory counts, unique constraints, sequence numbers, anything where a rule such as "never negative" or "at most one" must never be violated. These get a designated region that owns writes; other regions forward writes to the home and serve reads locally. The latency cost is real and unavoidable — the alternative is consensus, which costs the same round trip.

Class 4 — must not be replicated at all. Data with residency constraints, which is not a conflict problem but is often discovered during this exercise.

The mistake to name explicitly

Treating a balance or an inventory count as class 1 because the infrastructure offers last-writer-wins by default. Two concurrent decrements in different regions result in one being lost, the stock going negative in reality while positive in the database, and the error being invisible until a reconciliation months later. The infrastructure will not warn you; the default is a data-model decision presented as a configuration option.

Testing failover

  • Regularly, and in production. A failover path tested only in theory does not work; the standard evidence is that most failover mechanisms fail on their first real use.
  • Test failover and failback. Failback is harder — the failed region returns with stale data and must catch up without overwriting newer writes — and it is almost always less tested.
  • Test with real data volume, since replication lag and catch-up time are volume-dependent and the interesting failures only appear at scale.
  • Test the home-region entities specifically: when the home region is lost, who owns the invariant? Either writes to those entities are unavailable until the home region is reassigned — which is a legitimate, statable choice — or there is a promotion protocol that must guarantee no two regions ever believe they are home, which is a consensus problem and must be treated as one.
  • Measure and publish the data-loss window implied by asynchronous replication. RPO is not zero in an async multi-region design, and pretending otherwise is the most common documentation failure in this area.