practice

Test Data Management

The problem of getting data that is realistic enough to find defects and safe enough to exist outside production.

test-dataprivacytesting

This is the constraint that quietly determines how good an organisation's testing can be, and it is usually treated as an afterthought.

The two easy answers both fail. A copy of production finds real defects and is a privacy incident waiting for its disclosure date — it multiplies the number of systems holding personal data, and under most data protection regimes testing is not a lawful basis for processing it. Hand-crafted fixtures are safe and find nothing, because they have none of production's cardinality, skew, encoding oddities, historical inconsistency or scale.

The workable middle is masked or synthesised data that preserves statistical shape — the same distribution of values, the same proportion of nulls, the same long tail of customers with 4,000 orders and the ones with names that break your form validation. Masking must be referentially consistent across tables and irreversible, since a mask that can be joined back to identity has not de-identified anything.

The pragmatic architecture: synthesised data for the bulk of testing, a small tightly controlled production-like dataset behind real access controls for the specific cases that need it, and production data used only in production, by testing in production.