Testing Strategies

Definition

A testing strategy allocates effort across levels — unit, integration, contract, end-to-end, manual, production — to maximise confidence per unit of cost. The cost is not just runtime; it is maintenance, flakiness and the delay imposed on every change.

The shape and why

The pyramid — many fast isolated tests, fewer integration tests, very few end-to-end — reflects an economic reality rather than an aesthetic preference: as tests move up, they get slower, flakier, more expensive to maintain, and harder to diagnose when they fail. Each level should test what the level below cannot.

The common inversion — few unit tests, many end-to-end tests — produces a suite that takes 40 minutes, fails randomly, and is eventually ignored. Once a suite is routinely ignored, it provides no confidence at any cost.

What to test where

• Unit — business logic, edge cases, error handling. Fast, deterministic, no I/O.
• Integration — the code plus one real dependency: the database, the queue. Catches the mapping and configuration errors that unit tests with mocks cannot.
• Contract — the interaction between services, verified without deploying both. This is what replaces most end-to-end coverage in a distributed system, and it is the level most often missing.
• End-to-end — a handful of critical user journeys only. Expensive, valuable, and strictly rationed.
• In production — synthetic transactions, canary analysis, feature-flagged rollout. Some confidence is only obtainable here, and pretending otherwise produces enormous pre-production suites that still miss production behaviour.

The properties that matter more than the shape

Determinism. A flaky test is worse than no test: it trains the team to re-run rather than to investigate, and it eventually masks a real failure. Quarantine flaky tests immediately and fix or delete them.

Speed. A suite over about ten minutes changes behaviour — people batch changes, stop running it locally, and merge with it red. Speed is a correctness property indirectly.

Failure clarity. A failing test should say what broke. A test asserting forty things tells you only that something is wrong.

Testing behaviour, not implementation. Tests coupled to internal structure make refactoring expensive, which is the opposite of what tests are for.

Failure scenarios

• Coverage as the target, producing tests that assert the code does what it does.
• Mocking everything, so tests pass while the real integration is broken.
• An inverted pyramid with a slow flaky suite that is routinely bypassed.
• No contract tests in a distributed estate, so the choice is between end-to-end suites and hope.

Interview question

"Your end-to-end suite takes 40 minutes and fails randomly twice a week. What do you change?"