Shadow Comparison
Running a new implementation alongside the old on real traffic, comparing outputs without the new system affecting users.
The strongest correctness evidence available before a cutover, and the only technique that reliably finds the behaviour nobody documented.
The mechanism: real requests are sent to both implementations; the old system's response is served; both outputs are recorded and compared. The new system's errors are invisible to users.
What it finds that testing does not: undocumented behaviour that the old system exhibits and that someone depends on; edge cases in real data that no test dataset contains; and performance under genuine load and data distribution, which a synthetic test misrepresents.
The practical difficulties that must be planned for:
Side effects must be suppressed in the shadow path — no emails, no payments, no external calls, no writes to shared state. This is the part that requires real design work and is where shadow running goes wrong.
Non-determinism produces false differences — timestamps, generated identifiers, ordering. The comparison needs normalisation, or the difference report is noise.
Cost, since everything is computed twice.
Differences must be triaged, not just counted. A dashboard showing 3% mismatch is useless; a categorised list of difference types is what drives the work.
Reserve it for high-risk migrations — payments, pricing, regulated calculations — where the cost of a silent behavioural change is high.