Your 800-test end-to-end suite takes 90 minutes and fails roughly half the time for reasons unrelated to the change. The team re-runs until green. What do you do?

What the interviewer is testing

Whether you will do the arithmetic and reach the uncomfortable conclusion, rather than proposing to stabilise 800 tests.

The arithmetic

If each test independently passes 99.9% of the time — which would be extraordinarily good for end-to-end — then 0.999^800 ≈ 45%. A suite this size cannot be reliable unless each test is far better than anyone achieves in practice.

So the suite currently has veto power without detection power: it blocks merges, and because red is routinely wrong, nobody investigates red.

The action

Delete most of it. Not stabilise — delete. Keep the small set of revenue-critical journeys, in the region of 10 to 30, and treat their determinism as a hard requirement: any test in that set that flakes is fixed or removed the same week.

Push coverage down to where it fails deterministically and localises:

Quarantine before deleting if the team is nervous: move the suite out of the blocking path, observe for a month which failures were ever real, and delete on that evidence. In practice the number is very small, and that evidence is what wins the argument.

What a strong answer adds

Naming the cost that is not on anyone's ledger: an hour per failure per day spent deciding whether a red build is a defect, a data problem, a timing issue or an environment fault. That is the largest line item and it is invisible.

And pairing the reduction with production verification — synthetic journeys running continuously against production, which catch the same class of failure with better fidelity and no flakiness.

Common weak answers

Adding retries to the suite, which formalises unreliability. Investing months in stabilising 800 tests that will re-destabilise.