You virtualised a mainframe dependency for testing. Six months later a production integration fails that all tests passed. What happened?
Show the full answer Hide the answer
What the interviewer is testing
Whether you anticipated the permanent risk of any simulated dependency.
What happened
Drift. The virtual service encodes what the dependency did when it was recorded. The mainframe team changed a field, a code, an error condition or a timing behaviour, and the simulation did not change with it.
Every test passed against a world that no longer exists — producing confidence that is not merely useless but actively misleading, because the suite reported success.
The controls
Bind the simulation to a verified contract rather than to hand-written fixtures. If the provider verifies the contract in their own pipeline, a change that breaks it fails their build and you learn about it there.
Where the provider will not participate — which is common for mainframes and third parties —
Regenerate from recorded real traffic on a schedule, so the simulation tracks reality rather than being a snapshot.
Keep a small set of genuine integration tests against the real dependency, running periodically, whose only job is to detect that the simulation has gone stale. This is the essential control and it is the one that gets cut for cost.
Contract monitoring in production: validate real responses against the expected schema and alert on deviation, which catches provider changes at the moment they happen rather than at the next test cycle.
What a strong answer adds
The organisational fix that matters more than any of these: a change notification agreement with the dependency's owning team. Most such surprises are not malicious or even careless — the other team did not know you existed. Being a registered consumer with a named contact prevents more incidents than any technical control.
And noting that a useful virtual service reproduces latency, errors and rate limits, not just happy paths, since those are the behaviours calling code handles badly.
Common weak answers
Adding more virtual service scenarios, which does not address staleness. Blaming the mainframe team.