Netflix: Regional Evacuation
also called Chaos Kong, Region Failover
Netflix rehearses shifting all traffic out of an entire AWS region, which is what makes the capability real rather than documented.
The problem
Netflix runs active-active across multiple AWS regions. The value of that is entirely conditional on being able to move traffic when a region degrades — and a failover path that has never been exercised is an essay, not a capability.
The specific risks in a regional evacuation are not the routing change. They are: does the surviving region have the capacity to absorb the load, do caches in the receiving region survive the cold start, does the data replication lag matter, and does the process work when the people who wrote it are asleep.
What they did
They rehearse it. Chaos Kong exercises a full regional evacuation against production traffic, on a schedule, and the organisation treats the exercise as routine rather than as an event.
The architectural prerequisites are what make it possible: data replicated across regions asynchronously, services that tolerate serving from a replica, a routing layer that can shift traffic quickly, and enough standing capacity in the receiving regions to take the load — which means running below the utilisation a single-region deployment would target.
The trade-off
The cost is bought capacity that is idle in normal operation. Running three regions each able to absorb another's traffic means substantial headroom, permanently. Netflix pays this because a streaming outage is a subscription-cancellation event at scale.
The consistency cost is equally real: cross-region replication is asynchronous, so a failover can lose recent writes. Netflix's domain tolerates that — a lost "continue watching" position is an annoyance. A payment system's would not, and the same architecture applied to a ledger would be wrong.
The transferable lesson
Rehearsal is the difference between a recovery capability and a recovery document. Most organisations discover during a real event that the secondary region's quotas are sized for the pilot light, that replication has been broken for weeks with no alert, or that the runbook references a system that no longer exists.
You do not need Netflix's architecture to take the lesson. A timed, load-bearing failover exercise on a schedule finds the same defects at any scale — and finding them on a Tuesday afternoon is dramatically cheaper than finding them during the incident.