Incident Management Platform  ·  View 05 of 34  ·  2 · People and journeys

Journey — An Incident Commander Runs a SEV1 to Mitigation

From declaration to mitigated in 41 minutes. The trough is the split: a second, unrelated failure hiding inside an incident everyone has already acknowledged.

Editable source SVG draw.io All views
Incident commander checkout outage Goal — Stop user impact, keep everyone informed Trigger — Responder raises the incident to SEV1 Done when — Mitigated in 41 min, updates on time 1 · Declare 2 · Assemble 3 · Organise 4 · Split ◆ moment of truth 5 · Update 6 · Mitigated What they do Takes the IC role Pages database team Names scribe and comms Splits the DNS alerts out Posts the 30-min update Marks mitigated System behind it Severity obligations Service catalogue Role events Split event Update template Three clocks How it feels Calm Tense Stressed Where it hurts Who owns payments-db? Twenty people typing Second failure hidden Leaders asking in DMs What the platform gives Channel opened for SEV1 Page by service owner One holder per role New service re-pages Next-update-by timer Mitigated is not resolved Journey — An Incident Commander Runs a SEV1 to Mitigation v 1.0 · owner Reliability Architecture · date 2026-09

The moment of truth

  • DNS alerts for a different service grouped into the checkout incident because they arrived in the same minute. Nobody was paged for them. Grouping never crosses a service boundary on its own (ADR-11), and a split is an event that keeps both timelines whole.
  • A new service appearing in an acknowledged incident re-pages that service's rotation. The commander does not have to notice it first.

Decisions it relies on

  • Severity binds obligations: SEV1 opens a Mattermost channel, assigns an incident commander role, starts the stakeholder update clock and makes a review mandatory (ADR-13).
  • Roles are held by one person at a time and handed over as events, so the timeline says who was commanding at 03:40 without anyone reconstructing it.
  • Mitigated and resolved are different states with different clocks. The commander can stand the room down without claiming the cause is fixed.

Target

  • Stakeholder updates on the declared cadence, 30 minutes for SEV1, with the next-update-by time visible to the audience so silence is not read as neglect.
◀  Journey — An On-Call Responder Is Woken for a SEV1 at 03:12 All views Journey — A Rotation Owner Closes a Gap Nine Days Before DST  ▶
Solution Architecture v1.0 · Reliability Architecture · 2026-09 · 34 views · open source, fully on-premises05-journey-run-a-sev1-to-mitigation