An enterprise runs an annual disaster recovery test that always succeeds, yet the team has low confidence in real recovery. What is likely wrong with the test?

Why a passing test can mean nothing

Annual DR tests that always pass usually share a set of characteristics that remove everything difficult:

1. It is scheduled and prepared. Everyone knows the date. Systems are patched, configurations synchronised, and the standby environment is fixed in the weeks before. The test measures a state that exists only around the test.

2. It is a partial test. A subset of systems, or a failover without the failback, or a database promotion without moving actual application traffic. The parts that are hard are the parts excluded.

3. It uses a clean scenario. "The primary data centre is unavailable" — which is the easiest possible disaster. Real events are partial, ambiguous, and involve degraded rather than absent components.

4. Success is defined as completion, not as service. The runbook was executed; whether business transactions actually completed correctly in the recovered environment was not measured.

5. The people are the experts. The test is run by those who designed it. A real event at 3 a.m. involves whoever is on call.

6. Dependencies are out of scope. Identity, DNS, network connectivity, third-party integrations, licence servers — frequently excluded because they are "not our system", and frequently the reason real recoveries fail.

What a meaningful test looks like

• Unannounced, or announced with a window rather than a date.
• Includes failback. Returning, with data written during the event, is usually harder than failing over and almost never rehearsed.
• Measures business outcomes — can a customer order be placed and settled correctly in the recovered environment — rather than infrastructure status.
• Run by the on-call team, with the designers observing rather than participating.
• Includes at least one injected complication: a stale runbook step, an unavailable person, a dependency that does not come back cleanly.
• Produces defects. A test that finds nothing was not testing anything.

The enterprise-specific difficulty

The estate is large, the dependencies cross organisational boundaries, and a full test would affect business operations. That is a real constraint, and the workable answer is frequent partial tests with rotating scope plus an occasional broad one, rather than one annual exercise designed for a clean pass.

The question that exposes the gap

"What is our RPO, and how do we know?" If the answer comes from the replication configuration rather than from a measured restore, it is a design intention rather than a fact. The real RPO is the worst of all components — and the components most likely to be worse are the ones excluded from the test.