Postmortems

Definition

A postmortem records what happened, why, and what will change. Its purpose is organisational learning, not accountability, and those two purposes are mutually exclusive: an organisation that uses postmortems to assign fault will get postmortems that assign fault to nobody and explain nothing.

Why blameless is a mechanism, not a nicety

If an engineer believes a postmortem could damage them, they will describe the incident in a way that protects them. The account will be technically true and materially incomplete, and the actual cause — which is almost always systemic — will not surface.

The operating assumption is that people act reasonably given the information and constraints they had at the time. When someone runs a destructive command, the question is not why they were careless but why the system permitted it, why the consequence was not obvious, and why it was not recoverable. This is not generosity; it is the only way to get information that leads to durable fixes.

What a useful postmortem contains

• Impact in user terms. Not "the cache tier was degraded" but "34,000 users could not check out for 22 minutes".
• A timeline with detection, escalation, diagnosis and mitigation timestamps. The gaps in that timeline are usually the most valuable finding — often the largest is between the failure starting and anyone noticing.
• Contributing factors, plural. Serious incidents never have a single cause. The concept of the root cause is generally a stopping rule rather than a discovery.
• What went well. Including luck, explicitly labelled as luck, because you cannot rely on it twice.
• Action items with owners and dates, ranked, and honest about which will not be done.

Industry example

The blameless postmortem as a formal practice is most associated with Google's site reliability engineering, and the transferable insight is about prioritising the detection gap. Teams instinctively focus action items on preventing the specific trigger — which is often the least likely thing to recur — while the reusable improvements are in detection, diagnosis and mitigation speed.

Reducing time-to-detect from 18 minutes to 2 improves every future incident, including all the ones with causes you have not imagined. Preventing this exact trigger improves one.

Failure scenarios

• Action items that are all "add more monitoring", with no owner and no date.
• A single root cause identified, so contributing factors go unaddressed.
• Postmortems written and never read, with no mechanism for finding patterns across them.
• Only for large incidents, missing the near-misses that are the cheapest learning available.
• Blame present but unspoken, which is functionally identical to blame.

Interview question

"An engineer ran a command that deleted production data. What does the postmortem investigate?"