advanced 2 min answer

Your disaster recovery plan provisions capacity in the secondary region at failover time. Why is that a problem?

resiliencecontrol-planedr
Show the full answer Hide the answer

What the interviewer is testing

Whether you understand static stability and the reliability asymmetry between control planes and data planes.

The problem

Provisioning depends on the control plane, and control planes fail more often than data planes — in every system, including every cloud provider's. Study the major cloud outages and the failure is usually in a control plane: configuration propagation, a scaling subsystem, a registry.

So the plan requires the control plane to work at precisely the moment it is least likely to. Three specific failure modes:

The regional event that triggered failover may be affecting the control plane too.

Capacity may not be available. Everyone else failing over to the same region is requesting the same instance types simultaneously, and capacity is not guaranteed unless reserved.

Provisioning takes time — minutes to tens of minutes for a full stack — which is frequently longer than the stated recovery objective and is never included in the estimate.

Statically stable design

The failover target must already be running, sized for the load it will take, with data already replicating. Failover becomes a traffic change — a routing decision — rather than a build operation.

The applicable patterns:

Pattern Recovery time Cost
Backup and restore Hours Lowest
Pilot light (core running, rest provisioned on failover) Tens of minutes Low, and control-plane dependent
Warm standby (scaled-down full stack running) Minutes Moderate
Active-active Near zero Highest

Warm standby is the point at which the design becomes statically stable, because everything needed already exists and only scale changes.

The general principle

Do not put a control plane in a data plane's request or recovery path. The same reasoning applies to autoscaling during an incident, to fetching configuration or secrets per request, and to service discovery that fails closed when the registry is unreachable.

What a strong answer adds

Pre-scaling for the failure case rather than relying on autoscaling to react. If the secondary must take full load, it needs that capacity before the event — and testing failover under realistic load, not with synthetic traffic, is what proves it.

Common weak answers

Reserving capacity but still provisioning at failover. Assuming autoscaling handles it.