"Testing in production" is either professional practice or negligence depending on what is meant. What are the legitimate forms, and what must be true for each?

The distinction

Negligent: shipping untested code and observing whether users complain.

Professional: deliberately exercising production with controls, because production contains conditions no other environment reproduces — real data shapes, real traffic patterns, real dependency behaviour, real scale, real concurrency, and the accumulated state of years.

The legitimate forms all share three properties: the blast radius is bounded, the observation is deliberate, and the rollback is immediate.

The forms

Canary and progressive delivery. A new version serves a small, deliberately-chosen population with segmented, comparative health gates. The most established form.

Synthetic monitoring. Scripted transactions running continuously against production, exercising critical journeys. This is the only mechanism that detects a broken journey when no real user happens to be attempting it, and it is the foundation of external break-glass observability.

Feature flags with progressive exposure. The feature is deployed and disabled, enabled for internal users, then a percentage. Deployment and release are separated, which means the risky moment is a configuration change with instant reversal rather than a deployment.

Traffic shadowing. Real requests duplicated to a new implementation whose responses are compared and discarded. Nobody is exposed to the new system's errors and it is nonetheless tested against reality — with side effects being the hard problem that must be explicitly designed for.

Load testing in production, with tagged synthetic traffic, shadow writes and an abort threshold.

Chaos experiments, with a hypothesis, a bounded scope and a tested stop button.

Dark launches. New backend paths exercised by real traffic while the results are unused, so capacity and correctness are proven before anyone depends on them.

What must be true for any of them

• Observability sufficient to detect harm quickly, and specifically segmented rather than aggregate — a minority-segment failure is invisible in an aggregate by construction.
• A bounded blast radius, defined in advance.
• A rollback measured in seconds and tested, not assumed.
• Explicit organisational agreement, because the first visible impact will otherwise end the practice permanently.
• Isolation of side effects — synthetic traffic must not send emails, charge cards, decrement inventory or pollute analytics and training data.

Why it is not a substitute for testing before production

These techniques verify behaviour under real conditions. They do not verify logic, and a defect found in production has already affected someone even if the population was small.

The correct framing is layered: unit and contract tests catch logic and integration errors cheaply and early; production techniques catch the things that only exist in production — scale, data shape, dependency behaviour, and emergent interaction.

A team using production testing to compensate for absent test coverage is not doing this; it is doing the negligent version with better tooling, and the difference is visible in whether the defects being caught are logic errors or environmental ones.