advanced 2 min answer

A team has 85% line coverage and keeps shipping defects in tested code. What would reveal the gap?

digitmutation-testingcoverageassertionsquality
Show the full answer Hide the answer

What coverage does not tell you

Coverage measures whether a line executed during a test, not whether the test would fail if the line were wrong. A test that calls a function and asserts nothing produces full coverage and zero protection, and tests that assert weakly are extremely common — particularly ones written to raise a coverage number.

85% coverage with defects in tested code is the expected outcome, not an anomaly.

What reveals the gap

Mutation testing: deliberately introduce small changes to the code — invert a condition, change a boundary, remove a statement — and check whether any test fails. A mutation that survives is a line that is executed and unprotected.

The mutation score is a far more honest quality measure than coverage, and it is directly actionable: each surviving mutation names a specific missing assertion.

The cost and the scoping

It is computationally expensive, since each mutation requires running the affected tests. That makes whole-codebase mutation testing impractical for most teams.

The practical scoping: run it on the modules where correctness is expensive — the calculation logic, the state machine, the money paths — rather than everywhere. A high mutation score on the premium calculation matters; on the request-parsing layer it does not.

Run it on a schedule rather than per commit, and on changed code in the pipeline where the cost permits.

The insurance-domain application

Premium calculation, eligibility rules and claims adjudication are exactly the code where a wrong result is silent — the arithmetic works, the number is wrong, and nobody notices until a regulator or a customer does.

These are the highest-value targets for mutation testing, and they are also usually the best-covered by line coverage — which is precisely why coverage is misleading there.

The cheaper approximation

Review the assertions in the tests for the critical modules, by hand, once. A surprising proportion assert only that no exception was thrown, and finding them takes an afternoon rather than a compute budget.

Mutation testing automates and quantifies that review, which is its real value — but the manual version is available immediately and frequently finds the same things.