An organisation writes thorough postmortems and keeps experiencing the same class of failure. What is missing?
Show the full answer Hide the answer
What is missing
Almost always the action items, not the analysis. Postmortems produce a list, the list goes into a backlog, the backlog is prioritised against features, and the items are never done — so the analysis was an intellectual exercise.
The second thing missing is classification across incidents. A single postmortem explains one failure. Recurrence is a pattern, and patterns are only visible when incidents are categorised and reviewed in aggregate. Most organisations never do the aggregate review, so they solve each instance and never see the class.
What makes the loop close
- Action items with an owner, a date and a severity, tracked in the same system as feature work rather than in a document. If they are not visible in planning, they will not be done.
- A distinction between mitigations and fixes. A mitigation stops the current bleeding; a fix removes the possibility. Recording both, and being honest that only the first was done, prevents an organisation believing itself protected.
- Aggregate classification: contributing factor categories, affected components, detection method, time to detect, time to mitigate. Reviewing the aggregate quarterly is what turns incidents into architecture.
- A budget for reliability work that is not negotiated per item — an error budget, or a fixed share of capacity — because otherwise each individual action item loses to each individual feature, every time, and the aggregate outcome is nobody's decision.
- Detection as a first-class finding. "How long until we knew" is usually a bigger lever than "how long to fix", and it is under-examined because it is less interesting.
The cultural precondition
Blameless analysis, genuinely. Not as a stated value but as an observed one: if the postmortem identifies who made the change rather than why the system allowed it, engineers will describe events defensively and the analysis will be wrong. In a regulated financial business this is harder, because there is often a parallel process asking exactly the accountability question — and separating the learning review from the accountability process, explicitly, is what makes the learning review usable.
The question that redirects a postmortem productively
Not "what caused this" but "what would have made this a non-event?" That surfaces the systemic answer — isolation, a limit, a fallback, a gate — rather than the proximate one, and systemic answers are what stop the class rather than the instance.