What should a test strategy specify, and what makes strategies fail in practice?

What it should specify

1. What each layer is responsible for, and what it is not. Unit tests verify logic; component tests verify a service against its real database; contract tests verify interfaces; end-to-end tests verify critical journeys. Overlap wastes effort and, worse, creates the belief that something is covered when the responsible layer does not actually cover it.

2. What is deliberately not tested automatically, and how that risk is managed instead — through observability, progressive rollout, or accepted exposure. An unstated gap is assumed to be covered.

3. Ownership per layer. A test with no owner is not maintained, and an unmaintained test becomes flaky and then ignored.

4. Speed budgets per layer, since the strategy is worthless if the suite is too slow to run before a push.

5. What happens on failure — blocking, informational, or quarantined — because a failing test that blocks nothing trains everyone to ignore failures.

What makes strategies fail

• Written as an aspiration rather than reflecting what the codebase permits. A strategy requiring isolated unit tests in a system where nothing is isolable describes a different codebase.
• No enforcement, so it describes intentions while the suite drifts toward whatever is easiest to write — usually broad tests.
• Uniform coverage targets, which drive tests that assert nothing in areas where coverage is cheap and leave complex logic under-tested where it is expensive.
• Ignoring flakiness, which is the single fastest way to destroy the value of an entire suite.
• No production dimension, treating quality as a pre-release concern when some properties can only be verified live.

The risk-based allocation

Test effort should follow consequence, not code volume. The payment path, the permission check and the data-integrity invariant justify depth; a rarely-used administrative screen does not.

Uniform standards feel fair and misallocate the effort — which is why a strategy should state where depth is required rather than setting one bar everywhere.