advanced 2 min answer

A product team's test suite is slow and flaky, and removing tests causes escapes. What testing strategy resolves this?

testingpyramidcontract-testsflakinessnotionarchitecture-selection
Show the full answer Hide the answer

The diagnosis

The suite is shaped wrong: too many broad tests and too few narrow ones. Broad end-to-end tests are slow (they need everything running), flaky (any component or timing issue fails them), and unclear in ownership (a failure belongs to nobody).

Their value is real and their cost scales badly, so a suite dominated by them becomes something people re-run rather than investigate — at which point it stops catching anything.

The shape that works

A large fast layer plus a deliberately tiny slow one:

1. Unit tests for logic, running in milliseconds without infrastructure. Fast enough to run on every save.

2. Component tests for one service with its real database and stubbed collaborators. Catch integration defects within a boundary without needing the world.

3. Contract tests replacing most cross-service integration tests. Each consumer publishes what it needs; the provider verifies every consumer's contract in its own pipeline, so a breaking change fails the provider's build before merge, with unambiguous ownership.

This is the layer that replaces the expensive middle, and it works because it tests the actual constraint — not that everything works together in a lab, but that no service breaks what its consumers depend on.

4. A small number of end-to-end tests for critical journeys only — sign-up, the primary workflow, checkout. Not to catch interface breaks, which contracts handle, but integration assumptions no contract expresses: ordering, timing, configuration and data-state dependencies.

Handling flakiness

Treat a flaky test as a broken test. Quarantine it immediately so it stops eroding trust in the suite, and fix or delete it within a defined window. A suite with known-flaky tests trains everyone to re-run, which destroys the value of every reliable test in it.

Track flakiness explicitly — a rate per test — so the worst offenders are visible rather than tolerated.

The property that makes the shape sustainable

Feedback speed. A suite running in minutes is run before every push; one running in an hour is run nightly, and defects are found long after the change. The shape follows from optimising for feedback speed, which is why the fast layers must carry the load.