A payments company must build a test dataset for its fraud-scoring service. Production holds 400M transactions with about 0.2% labelled fraudulent; legal will not permit production data in test environments. The team needs the dataset to exercise the model's decision boundary and the pipeline's handling of awkward records. Which approach fits the requirement as stated?

The deciding property

The dataset has to reproduce the joint distribution, not the field-level validity. A fraud model's decision boundary lives in correlations: amount against merchant category, hour of day against geography, velocity against account age. A generator that produces individually valid records with independently sampled fields destroys exactly the structure the model was trained on, which is why a dataset can be large, clean and completely useless.

A model fitted to production and sampled from it preserves those correlations, including the 0.2% class imbalance and the long tail of odd-but-real records. It also produces records that correspond to no individual, which is what the legal constraint requires.

What it costs and how it can go wrong

• It can memorise. A generative model trained on rare records can emit something close to a real one, which is a re-identification risk rather than a privacy solution. Mitigate with a differentially private training procedure, or at minimum a nearest-neighbour check between generated records and the training set before release.
• It reproduces yesterday's distribution. Fraud patterns shift; a generator frozen in March produces a March world. Refresh it on a schedule, and keep the refresh date on the dataset.
• It is not evidence of production behaviour. Synthetic data validates pipeline and model mechanics. Final performance numbers still come from a held-out production evaluation inside the controlled environment.

Why the other options fail

• Rule-based generation across every field combination. This is the standard mistake. It produces a uniform world where every merchant category is equally likely at every hour, so the model's features carry no signal and the tests pass on data no fraud model would ever see. It is a reasonable choice for schema and load testing, and a poor one for anything about behaviour.
• A masked copy with format-preserving tokens. Masking keeps the correlations, which is its appeal, and it does not remove the obligation: the records still describe real people, and the quasi-identifiers that remain — amount, merchant, timestamp, postcode — are usually enough to single someone out. Legal's objection was to production data, and a masked copy is production data with substitutions.
• A uniform 1M-row sample with identifiers dropped. Two problems compound. It is still personal data, and at 0.2% base rate the sample carries roughly 2,000 fraud cases, with the rarest patterns — the ones that matter — likely absent entirely. Stratified sampling fixes the second problem and not the first.

When this is the wrong answer

If the requirement were only "exercise the pipeline", the rule-based generator is cheaper, faster and sufficient, and fitting a generative model would be waste. The generator earns its cost when the thing under test learns from the distribution. That is the flip condition worth remembering: model-shaped tests need distribution-shaped data; plumbing-shaped tests do not.