An end-to-end suite takes 40 minutes and fails randomly twice a week. What do you change?

What is being tested

Whether you recognise an inverted test pyramid and know what replaces the coverage you are removing.

First: quarantine the flakiness immediately

A flaky test is worse than no test. It trains the team to re-run rather than investigate, and once that habit exists a genuine failure is dismissed as flakiness. That is a real production incident waiting to happen.

So: move flaky tests out of the blocking suite today, into a quarantine that is tracked and visible. Then fix or delete them within a defined window. Deleting is a legitimate outcome — a test nobody trusts provides no confidence at any cost.

Then: diagnose the shape

Forty minutes and random failures is the signature of an inverted pyramid — few unit tests, many end-to-end tests. As tests move up the pyramid they get slower, flakier, more expensive to maintain and harder to diagnose. A suite made mostly of the top level inherits all of that.

The economics are unforgiving: a suite over about ten minutes changes behaviour. People batch changes, stop running it locally, and eventually merge with it red. At that point it provides no confidence while costing everything.

The redistribution

1. Push logic coverage down to unit tests. Most of what the end-to-end suite verifies is business logic that can be tested in milliseconds with no I/O. This is the bulk of the work and the bulk of the payoff.

2. Add contract tests for service interactions. This is the level most often missing and the one that replaces most end-to-end coverage in a distributed system: each consumer declares what it uses, the provider verifies against it in its own pipeline, and a breaking change fails a build rather than an environment. Integration confidence without deploying everything together.

3. Keep a handful of end-to-end tests — the three to five critical user journeys, and no more. They are expensive and valuable, and they should be rationed accordingly.

4. Move some confidence to production. Synthetic transactions, canary analysis with automated comparison, feature-flagged rollout. Some things are only observable in production, and enormous pre-production suites that try to substitute for that still miss it.

Fixing the causes of flakiness

Almost always one of: shared mutable state between tests, timing assumptions and sleeps, test order dependence, real external dependencies, or non-deterministic data. Each has a standard remedy — isolation per test, explicit waits on conditions, randomised ordering to expose dependence, stubbed externals, and controlled fixtures.

What to measure afterwards

Suite duration, flake rate, and — the one that matters — how often the suite catches a real defect that would otherwise have reached production. If the answer is rarely, the suite is ceremony and the redistribution is overdue regardless of its speed.