When should external dependencies be virtualised in testing, and what must the virtual service reproduce?

When to virtualise

Whenever the dependency is outside your control — a third-party API, a partner system, a payment provider, a carrier. Testing against the live service produces tests that fail for reasons you cannot fix, cannot run locally, cost money per call, and may have rate limits.

And especially for failure testing, which is the case that matters most: you cannot make a real provider fail on demand, so the only way to test your behaviour when it does is to simulate it.

What the virtual service must reproduce

Not just the happy path. The valuable virtualisation reproduces the behaviours that cause real incidents:

1. Slow, not down. The most damaging and least tested mode. A dependency responding in eight seconds instead of two hundred milliseconds holds threads and connections while error-rate circuit breakers do nothing, because nothing is erroring.

2. Partial and truncated responses — valid-looking data with fields missing. Tests whether your adapter defaults silently, turning a provider problem into an unattributable correctness problem, or rejects explicitly.

3. Intermittent failure, alternating success and failure, which is what actually makes circuit breakers flap and retries amplify.

4. Rate limiting, so backoff behaviour and Retry-After handling are exercised.

5. Malformed and unexpected responses, including error formats the documentation does not mention.

6. Correlated failure across several dependencies, testing whether the aggregate deadline and thread pools hold.

What to assert

Not merely that nothing crashed:

• The global deadline was respected and partial results returned.
• The failing dependency was isolated — others' latency and success rates unaffected. This is the most valuable assertion, because per-dependency bulkheading is what prevents one provider's bad day becoming an outage.
• Telemetry attributed the failure correctly.
• No incorrect data was published from partial or stale input.

The contract question

A virtual service is a guess about the real one. It must be kept honest — by recording real interactions, by contract testing against the provider where possible, and by periodic verification against the live service in a controlled environment.

A stub that has diverged from the real dependency gives confidence without assurance, which is worse than no test.