Diagnostic Radius
also called Failure Blast Radius, Signal Narrowness
The number of distinct causes consistent with a test's failure - the property that actually separates the layers of a test suite and decides what a red build costs to investigate.
A suite is usually described by what its tests execute: units, services, journeys. That framing explains neither why a broad suite is expensive nor why teams stop trusting it.
The useful property is how many causes a failure leaves open. A failing direct call implicates one function or the test itself. A failing browser journey implicates that function plus routing, serialisation, authentication, the database and its migration state, seeded data, the container clock, a port collision and the network. Same red bar, ten times the investigation.
Why it matters
The cost of a test is not its runtime. It is runtime plus the expected cost of diagnosing its failures, and the second term scales with the radius while the first scales with the machine.
That explains the behaviour everyone observes. A 3-second test runs on every save, so a defect is found while the change is still in working memory. A 40-minute suite runs after merge, so the same defect costs a context reload and a partial revert. It also explains why suites decay: a broad test that fails for eleven different reasons trains people to rerun rather than read, and a rerun culture is the beginning of a suite nobody believes.
Implementation patterns
- Classify by radius, not by tool. A test that starts a process but stubs every network call has a small radius despite looking like an integration test.
- Count causes per failure. Take the last quarter of red builds for a broad test, label each root cause, and if one cause dominates, write a narrow test for it and delete that coverage from the broad one.
- Budget the broad layer by journeys, not by features. One smoke test per critical journey is worth a great deal; the four hundredth end-to-end test is worth very little and pays full price.
- Make failures narrow after the fact where you cannot narrow the test: per-dependency health assertions that run first, so "the database container never became ready" is reported as that rather than as a business assertion failure.
- Keep a fast lane that runs on save in under a minute, so the narrow layer is where daily work happens.
Industry example
Etsy's published move to continuous deployment from 2011, with many deploys a day, is the clearest demonstration of the economics: a gate that runs on every change must produce a signal an engineer can act on in minutes, which is a statement about radius rather than about coverage. The same reasoning drives the "unit tests on save, integration on push, journeys on release" ladder that most delivery pipelines converge on in production.
Failure scenarios
- The rerun reflex. A broad test fails, the build is rerun, it passes, nobody learns anything, and a real regression is eventually rerun away.
- Mock-driven narrowing. A team converts integration tests to mocked unit tests to speed the suite, and the bug class the tests existed for moves silently to production.
- Attribution blame. A shared broad suite fails for an unrelated team's change, and the cost is paid by whoever merged next.
- The 40-minute local suite, which is not run locally and so stops preventing anything.
Trade-offs
| Choose | Gains | Pays |
|---|---|---|
| Narrow tests | Fast feedback, precise diagnosis, cheap parallelism | Blind to wiring: serialisation, dialects, middleware, migration order |
| Broad tests | Covers the integration where real defects hide | Slow, flakier, and each failure costs an investigation |
The resolution is not a ratio handed down as policy. It is a small number of broad tests chosen for the journeys that must never break, and narrow tests for everything whose failures are usually local.
When not to use it
Do not use radius as an argument for deleting integration coverage of the integration itself. If the risk you are protecting against is the SQL dialect, a fake will agree with you forever, and trading that test for a fast one buys a green dashboard with real defects. The rule applies to choosing where a given behaviour is tested, not to dropping behaviours that only exist between components.
Interview question
Q: A team's end-to-end suite takes 40 minutes and fails about a fifth of the time. They propose converting most of it to unit tests with mocks. Where do you agree, where do you push back, and what evidence would you gather first?
What a strong answer covers: the failure-cause census as evidence rather than opinion · agreement where failures are dominated by narrow causes · pushback where the tests cover wiring that a mock reproduces incorrectly · the option of keeping fewer broad tests rather than lower-fidelity ones · and the feedback-loop argument that the point of the change is running the suite before pushing.
Quick check
Quiz: Two tests cover the same line; one is a browser journey. What differs? The set of causes consistent with a failure, which sets the investigation cost.
Flashcard: When should a test move down a layer? — When a census of its recent failures shows one narrow cause dominating, never merely because the suite is slow.