Characterisation Test

A conventional test asserts intended behaviour, derived from a specification. A legacy system frequently has no usable specification, and the behaviour that matters is whatever it currently produces — because that is what its consumers have adapted to over years.

A characterisation test captures real inputs and the outputs the system currently produces, and asserts that they continue to match. It makes no claim that the behaviour is correct. It claims only that it is current, which is precisely what a refactoring or migration must preserve.

Why it matters

Without it, no change to a legacy system is safe, and every proposed improvement is a gamble whose downside is discovered by customers. That is why untested legacy systems calcify: the rational individual response to "this code is frightening and untested" is not to touch it, and the accumulated effect is a system nobody can change.

The subtler point is that you cannot yet know which of the system's bugs are depended upon. A rounding quirk, an off-by-one in a date range, an unusual null handling — some of these are latent defects and some are load-bearing, and consumers have built around them. Characterisation tests preserve all of them until you can tell the difference, which is the correct default.

Implementation patterns

• Capture real production inputs, sampled to cover the range of shapes, not synthetic ones — the value is entirely in the cases nobody would have invented.
• Record outputs as-is and assert equality, rather than asserting properties. Approval-style tests, where a stored expected output is compared and a change is explicitly reviewed and accepted, fit this exactly.
• Start at the coarsest boundary available — the whole system, or a whole request — and refine inward as seams appear. Coarse coverage first is what makes the first refactoring safe.
• Include the ugly cases deliberately: nulls, empty collections, historical records with obsolete shapes, the largest and smallest values present in production.
• Normalise genuinely non-deterministic elements — timestamps, generated identifiers, ordering of unordered collections — with an explicit list, since a suite that fails randomly is disabled within a week.
• Treat a failure as a question, not a defect: did the behaviour change intentionally? If so, review and update the recorded output deliberately.
• Scrub personal data from captured inputs, since production data in a test fixture is a recurring source of breaches.
• Track coverage of behaviour, not of lines — the useful measure is what fraction of real production traffic shapes are represented.

Industry example

The technique is central to the standard legacy-modernisation literature and to every large strangler migration: before extracting a component, its current behaviour is recorded so that the extracted version can be proven equivalent. In practice it converges with traffic shadowing — running real production traffic against both the old and new implementations and comparing — which is characterisation testing performed continuously in production rather than in a suite.

The recurring finding in such migrations is the same one: a meaningful minority of divergences turn out to be undocumented legacy behaviour that something depends on, and without a mechanism that records current behaviour those dependencies are discovered by breaking them.

Failure scenarios

• Asserting intended rather than current behaviour, which fails immediately and abandons the exercise.
• Synthetic inputs only, missing the cases that carry the risk.
• Non-determinism unhandled, producing random failures and rapid abandonment.
• Treating every failure as a regression, so intentional changes become a fight with the suite.
• Personal data captured into fixtures, creating a compliance problem.
• Fine-grained tests written first, coupling the tests to the structure being refactored — which is exactly backwards, since the structure is what is about to change.
• The suite kept forever, long after the migration, when many of these tests should be replaced by intention-revealing tests once the behaviour is understood.

Trade-offs

Characterisation tests encode bugs as expected behaviour, which is uncomfortable and is the point. They give no guidance about correctness, they must be reviewed carefully whenever they fail, and a large suite of them is a maintenance burden that resists intentional change as effectively as it resists accidental change.

They are also coarse and slow by nature — full-system, real-input tests are not unit tests — and they provide poor localisation when they fail, telling you that something changed rather than what.

The trade is precision and elegance in exchange for the ability to change a system at all. For a well-specified, well-tested system they are unnecessary. For an untested legacy system they are the prerequisite for every other technique, and skipping them is why so many modernisations either stall or break things nobody knew existed.

Interview question

"You have inherited a fifteen-year-old billing system with no tests and you must change it. Tell me what you write first, where the inputs come from, and what you do when one of your new tests locks in behaviour you are fairly sure is a bug."