A team adopts mutation testing to stop shipping defects in code that coverage says is tested. On a 40,000-line service with a 4-minute unit suite, what have they bought, what are they paying, and when does that bill arrive?

What is gained, quantified

Coverage answers "was this line executed". Mutation testing answers "would the suite have failed if this line were wrong", by changing the code in small ways — flipping a comparison, removing a call, altering a boundary — and checking whether a test notices. The gap between the two is routinely large: a suite at 85% line coverage commonly kills 50-65% of mutants, and the surviving mutants point at the specific assertions that are missing rather than at a percentage.

That is the purchase: a defect-shaped signal rather than a reach-shaped one, localised to a line and a mutation you can read.

What is paid

The cost is multiplicative, and this is the number teams do not run before adopting. Each mutant requires running the tests that cover the mutated line. A 40,000-line service generates on the order of several thousand mutants; even with test selection, per-mutant runs at a few seconds each put a naive full analysis in the hours range against a 4-minute suite. Tools reduce this with coverage-based selection, parallelism and incremental analysis, but the shape holds: mutation analysis is one to two orders of magnitude more expensive than the suite it analyses.

The second cost is triage. Some surviving mutants are equivalent — the mutated program behaves identically, so no test can kill it — and deciding that takes a human reading code. Equivalent mutants are the reason mutation scores are never driven to 100%, and a team that treats the score as a target burns its time on them.

When the bill arrives

Not on adoption, when someone runs it once on a laptop and the findings are genuinely useful. It arrives when it is wired into the pipeline on every pull request, at which point the feedback loop goes from four minutes to forty and developers start batching changes. A slow pipeline changes how people work, and that cost is paid by every change, while the benefit accrues only where assertions are weak.

Keeping the option to reverse

The scoping decision is what makes this sustainable:

• Run mutation analysis on the diff, not the repository. Mutate only lines the pull request changed, which is typically tens of lines, and the run fits in a minute.
• Run the full analysis on a schedule, weekly or nightly, as a report rather than a gate.
• Scope by consequence. Pricing, permissions, money movement and retry logic justify it; serialisation, configuration loading and glue code rarely do.
• Gate on the delta. "No new surviving mutants in changed lines" is enforceable. "Mutation score above 80%" rewards deleting hard tests and provokes argument about equivalent mutants.

When not to bother

If the suite's problem is that it does not execute the code at all — coverage in the 30s, whole modules untested — mutation testing tells you what you already know at high cost. Fix reach first. Mutation testing is the tool for a suite that is broad and shallow, which is the state a coverage target reliably produces, and that is also why the two often arrive in the same quarter: the gate created the problem the new tool detects.