What does it mean to think in failure modes as a habit rather than as a checklist item, and which questions consistently produce findings?

The shift in default question

Most design conversation asks "how does this work?" Failure thinking asks, continuously, "what happens when each part of this does not?"

The distinction is not a review stage; it is a habit applied while the design is being formed, when changing the answer is free. A failure mode identified during design is a paragraph; the same one identified in production is an incident.

The questions that consistently produce findings

• "What happens when this dependency is slow rather than down?" The single most productive question in the discipline, and consistently unconsidered. Teams design for failure and not for degradation, and degradation is both more common and more damaging — because retries amplify it and health checks do not detect it.
• "What is the blast radius when this fails completely?" Any answer of "everyone" is the finding.
• "What does the recovery require, and does the failure prevent it?" A failure that destroys its own remediation path — a kernel driver, a config service, a network device configuration — is a different risk class entirely.
• "What happens when this is called twice?" Idempotency, surfaced before it is a duplicate charge.
• "What happens when these arrive out of order?"
• "What is the state after a crash exactly here?" Asked at each point between two writes, this finds every dual-write problem.
• "How would we know?" A failure mode with no detection is one that runs until a customer reports it.
• "What does this look like at 10× and at 0.1×?" Both directions: empty states and cold caches break systems as reliably as load does.
• "Who is paged, and what do they see?" Which surfaces missing observability and missing runbooks together.
• "What are we assuming that we have not verified?" Assumptions are where failures live, and they are invisible precisely because they are assumed.

The dispositions that make it work

• Assume everything fails, including the things that never have. A component with a perfect record has either been lucky or has not yet met its condition.
• Distinguish independent from correlated failures. Redundancy protects against the first and not the second, and most real incidents are correlated — a shared dependency, a shared deploy, a shared configuration, a shared assumption.
• Follow the chain. A failure is rarely one thing; the interesting question is what the response to the first failure does to the second.
• Look for positive feedback. If being behind generates more work, the system does not recover when load falls — which distinguishes a slow system from a collapsing one and is the property that turns an incident into an outage.

Avoiding the failure mode of failure thinking

It can become paralysis, or theatre. Every system has infinite failure modes and most do not matter.

The discipline is prioritising by consequence rather than by likelihood, since likelihood is unknowable and consequence is estimable. A high-consequence failure with an unknown probability deserves attention; a low-consequence one with a known probability usually does not.

And the output must be a decision. For each identified mode: prevent it, detect and mitigate it, or accept it explicitly and write that down. Accepting a risk knowingly is a legitimate and frequently correct outcome — the failure is accepting it without noticing.