concept

End-to-End Test Cost Curve

How the cost of a broad end-to-end suite grows with its size while its marginal value falls, and where the two cross.

End-to-end tests have the highest confidence per test and the worst scaling properties. Cost grows faster than linearly: each test adds runtime to a sequence that is hard to parallelise, adds environment and data requirements, and adds a flakiness probability that compounds — at ninety-nine percent per-test reliability, a two-hundred-test suite fails spuriously more often than it passes.

Value grows sub-linearly, because tests overlap. The tenth end-to-end test traverses much of the same routing, authentication and configuration as the first nine, and only its final assertion is new.

The crossing point is lower than teams expect — usually a few dozen tests covering the critical journeys. Past it, each addition makes the suite slower and less trusted, and a suite that is not trusted is not a gate. The discipline is to cap the count deliberately, keep only journeys whose failure would be a serious incident, and push everything else down to contract and integration levels where it costs a fraction as much.