advanced
2 min answer
A platform runs three replicas of every service across three availability zones and still experiences total outages. What kinds of failure does that redundancy not address?
Show the full answer Hide the answer
What replica redundancy addresses
Independent failures: a host dying, a disk failing, a zone losing power. For those, three replicas across three zones is genuinely effective.
What it does not address
- Correlated failures from a shared cause. A bad deployment reaches all three replicas. A poisoned configuration reaches all three. A memory leak affects all three at the same rate. Redundancy multiplies copies of the same software, so a software failure is not made less likely by having more of it.
- Shared dependencies. Three replicas calling one database, one cache cluster, one identity provider have one failure domain regardless of their own count.
- Overload. Three replicas at capacity fail together, and losing one shifts its load to the others, which accelerates the failure — the classic cascading pattern where redundancy makes the collapse faster rather than preventing it.
- Data corruption. Replicated data is corrupted in all replicas, quickly and faithfully. Replication is not backup, and this is the confusion that produces the most complete data losses.
- Control-plane failure, which affects everything simultaneously and is not made better by more instances.
What actually addresses these
- Staged rollout with automated halt conditions, which is the primary defence against the correlated software failure — the most common cause of total outages in mature systems.
- Cell-based architecture, partitioning customers into independent cells with their own full stack, so a bad change or a poisonous request affects one cell. This is redundancy applied to the failure domain rather than to the instance count.
- Capacity headroom that accounts for failure — sizing so that losing a zone leaves enough capacity, rather than sizing to the total.
- Backups with verified restore, which is the only answer to corruption.
- Static stability, so control-plane failure does not stop serving.
The question to ask of any redundancy claim
"What is shared?" Three replicas sharing a deployment pipeline, a configuration source, a database and a codebase are three copies of one system. The redundancy is real for the failures that are independent and absent for the ones that are not — and the outages that make the news are almost always the second kind.