Review this. Legal forbids production data in test environments so a team trains a generative model on the production database and emits 200 GB of synthetic records nightly. Privacy sign-off says no real records are used. Last week an engineer recognised a real customer name in a test fixture. What would you change?

What is actually required

Two properties, and the team has been treating them as one: the dataset must be legally safe to hold outside production and shaped like production, including the parts of production nobody designed. A generative model is a reasonable tool for the second. It is not by itself a mechanism for the first, and the sign-off has confused the two.

The defect

A model trained on records can emit them. Memorisation is a documented property of generative models trained on data containing rare, high-entropy strings - exactly what names, account numbers and addresses are - and extraction attacks against language models have been demonstrated repeatedly since 2021. The recognised customer name is not an anomaly to be explained away; it is the expected behaviour showing up, and the only reason you heard about it is that an engineer happened to recognise one name out of 200 GB.

The structural problem is that the privacy claim is architectural rather than measured. "We generate rather than copy" is an argument about the pipeline. What a regulator and an incident reviewer both need is a property of the output, tested on every batch.

The one change that matters

Add a leakage gate to the generation job and fail the nightly run on a hit:

• Exact and near-duplicate match of every emitted record against the training corpus, on the identifying fields, with a similarity threshold rather than equality - a name with one character changed is still a disclosure.
• A membership-inference check on a held-out sample: if a classifier can tell training rows from held-out rows using the generator's outputs, the generator is leaking and the number tells you how much.
• Record the result with the dataset. The dataset's clearance is the gate's verdict, not a one-time sign-off from last year.

If the obligation is strict, train under a differential-privacy budget and publish the epsilon with each batch. That buys a provable bound and it costs the tails - a tight budget suppresses precisely the rare combinations that break code - so expect to keep two datasets with different clearances rather than one that satisfies both needs.

What I would leave alone

• Generating nightly at 200 GB. Volume and freshness are not the problem and the query planner needs the volume.
• The decision to generate rather than mask. A masked production dump carries the same obligations as production, and masking destroys the malformed historical rows that find bugs. Generation is the better bet, correctly governed.
• The model itself, for the bulk of the distribution. Its weakness is the tail, which brings the second change: keep a small hand-curated corpus of awkward records - the 2015 row with a null currency, the customer with 400 addresses - under the same controls as production data, and run it as a separate fixture. A generator fitted to the bulk will not invent your worst records, and those are the ones that break migrations.

How I would argue this - and when generating is the wrong answer

Not as a privacy objection, which invites a debate about likelihood. As a testability one: the current design makes an unfalsifiable claim, and two cheap automated checks turn it into a number that can be shown to an auditor and watched over time. The gate is also the thing that lets the team keep the generator, which is the outcome everyone in the room wants.