A payments platform needs realistic test data. What are the options, and what must be true regardless?

The options and their trade-offs

• Masked production data. Most realistic in shape, volume and distribution — which is what makes it valuable, since load-dependent and data-dependent failures live in exactly those properties. The risk is that masking is incomplete, and production data in a non-production environment is a recurring source of breaches.
• Synthetic data generated to match production's statistical properties. Safe, and only as good as the generator's model of the real distribution — which typically misses the anomalies that cause the bugs.
• Curated fixtures for specific scenarios. Precise, deterministic, excellent for regression testing, and useless for anything volume- or distribution-dependent.
• Recorded production traffic replayed, which is realistic for behaviour and carries the same privacy problem as masked data.

What must be true regardless

• Masking automated, not procedural. A policy that someone must remember to follow will be forgotten, and the failure is silent. The masking should be part of the copy pipeline with no path that produces unmasked data.
• Referential integrity preserved through masking, or the data is realistic in shape and broken in relationships — which produces tests that fail for reasons unrelated to the code.
• The anomalies retained. Real data contains records created before a constraint existed, records fixed by hand, and encodings nobody documented. Those are precisely what breaks a migration or a new code path, and a sanitising process that removes them removes the value.
• Deterministic and resettable, so a test's failure is reproducible.

The payments-specific requirement

Test data must exercise the state machine's unusual paths: the partial refund, the chargeback after settlement, the authorisation that timed out and was reconciled, the duplicate with the same idempotency key.

These are rare in production and are where the correctness risk is, so a dataset sampled proportionally from production will contain almost none of them. Curated fixtures are the right tool for that class, used alongside volume data rather than instead of it.

The combination that works

Curated fixtures for correctness, masked or synthetic volume data for performance and migration, recorded traffic for behavioural regression. Each answers a different question, and using one for all three answers none of them well.