An erasure request arrives for a customer whose data has propagated into a warehouse, a lake, search indexes, caches, backups and trained models. How is this handled, and what should have been designed in advance?
Show the full answer Hide the answer
Why this is hard after the fact
Personal data propagates by default. It is copied into analytical stores, denormalised into derived tables, indexed for search, cached for performance, exported to partners, included in backups, and used to train models. Each copy is created by a pipeline whose author was not thinking about erasure, and no inventory of the copies exists.
The request is therefore not "delete a row" but "find every derivative of this person across an estate nobody has mapped."
The mechanisms, by store type
- Operational databases: delete or anonymise, straightforwardly, provided the record can be located across every service that holds it.
- Analytical stores on open table formats: a delete produces a new version; the old version remains until snapshots are expired, so erasure requires expiry as well as deletion — a step routinely missed.
- Search indexes: deletion by document, plus verification that nothing was rebuilt from a stale source.
- Caches: invalidation, with TTL as a backstop.
- Backups: the genuinely hard case. Rewriting backups is impractical and frequently impossible, so the usual position is that backups age out on their retention schedule and are not restored selectively, with a documented process to re-apply erasures after any restore. That process must exist and be tested, or a restore silently resurrects erased data.
- Trained models: the data cannot be extracted from the weights. The practical positions are retraining on a schedule that ages out the individual's contribution, or arguing that the model does not constitute personal data — a legal position rather than a technical one, and it must be taken deliberately.
- Partner exports: contractual obligation to propagate the erasure, and a record of what was sent to whom.
What should have been designed in advance
- Crypto-shredding. Personal data encrypted under a per-subject key, so erasure is destroying the key and the ciphertext may remain in every log, backup, replica and archive — reaching places rewriting cannot. The discipline required is knowing exactly what is inside the envelope, and the partition key is the most common leak.
- A subject identifier that is traceable, so every copy can be found — which requires it to be present and consistent rather than transformed differently in each pipeline.
- Lineage capture, so the derivatives of a source table are enumerable rather than reconstructed by asking people.
- Data classification at ingestion, so which columns are personal is recorded rather than inferred later.
- Minimisation: not copying personal data into analytical stores at all where a pseudonym or an aggregate would serve. The cheapest erasure is the copy that was never made, and this is the highest-value control by a wide margin.
- A tested erasure runbook, with a measured completion time and a verification step.
The verification nobody does
Confirm the erasure actually happened, by attempting to retrieve the subject across every store afterwards. An erasure process that is assumed to work is a compliance claim with no evidence behind it, and the failure is discovered by a regulator or by a customer finding their data still present.