You are incident commander. Error rate is 8% and rising, cause unknown. What do you do in the first ten minutes?
Show the full answer Hide the answer
What the interviewer is testing
Whether you prioritise mitigation over diagnosis, which is the distinguishing judgement of good incident response.
The first ten minutes
Declare the incident and take the role explicitly. Ambiguity about who is coordinating is what makes the first twenty minutes chaotic. State it.
Mitigate before diagnosing. This is the core discipline. The question is not "what is wrong" but "what makes this stop":
Did anything change recently? Deployments, configuration, flags, migrations, and — commonly forgotten — changes by other teams and provider-side changes. Reverting a recent change is the fastest mitigation available and it does not require understanding the fault.
Can we shed or degrade? Disable a feature, shed non-essential load, fail over.
Can we reduce blast radius? Route affected traffic away.
Assign roles. Someone investigating, someone communicating, someone executing mitigations. The commander coordinates and does not debug — a commander who starts debugging stops commanding.
Communicate early with what is known. Customers and internal stakeholders need acknowledgement before they need a cause. "We are aware of elevated errors, investigating, next update in 15 minutes" is sufficient, and the commitment to a next update is the part that matters.
Start a timeline as you go. Reconstructing it afterwards is slow and inaccurate.
What not to do
Do not spend the first ten minutes on root cause. Understanding can come after users stop being affected. Teams that invert this routinely spend an hour diagnosing while a rollback would have fixed it in five minutes.
Do not make multiple simultaneous changes — you will not know which helped, and one may make it worse.
What a strong answer adds
Time to detect as the metric to examine afterwards. If the first signal was a customer complaint, detection failed regardless of how well the response went — and detection is usually the largest and most reducible component of total incident duration.
And the hardest judgement: when to stop mitigating and start diagnosing, which is when the user impact is contained rather than when the cause is understood.
Common weak answers
Diagnosing root cause first. Waiting for certainty before communicating.