advanced 2 min answer

Your platform is built on an append-only event log with a lakehouse behind it. Legal asks how you will satisfy erasure requests within 30 days. What is your answer?

privacyerasurearchitecturecompliance
Show the full answer Hide the answer

Establish the actual scope first

Erasure applies to personal data held about the subject, and it has exceptions — data retained under a separate legal obligation, such as financial records with a statutory retention period, is generally out of scope. Getting the scope agreed with legal before designing is what stops this becoming an unbounded programme.

Then map, from lineage rather than from memory, every location a subject's data reaches: the event log, raw tables, modelled tables, snapshots, search indexes, caches, logs, backups, and every third party the data was pushed to.

That list is the problem. Deleting from the primary store is the easy part.

The mechanism: separate identity from event

Retrofitting deletion into an immutable log is close to impossible. The workable architecture separates the two:

  • Events hold a subject reference, not personal data.
  • Personal data lives in a mutable subject store, keyed by that reference.
  • Erasure deletes from the subject store. The events remain — they are facts about what happened, and they carry no identifying content once the reference no longer resolves.

Analytical aggregates built from events survive erasure, which is usually the outcome the business needs and legal accepts.

Where the log already contains personal data

If the events themselves carry names and addresses, the options are narrower:

  • Cryptographic erasure — encrypt per subject and destroy the key, rendering the ciphertext permanently unreadable everywhere, including in every backup and snapshot, without touching them. Widely accepted, though confirm rather than assume it in your jurisdiction. It must be designed in; retrofitting means re-encrypting the estate.
  • Rewrite by replay — reprocess the log excluding the subject and swap. Correct, and expensive enough that it is usually reserved for a small number of cases.
  • Tombstones with compaction on a keyed topic, which works where the log is a changelog rather than a fact stream.

Backups and snapshots

State the position explicitly rather than leaving it implicit: backups are excluded from the immediate deletion and age out within a stated retention period, during which they are not restored for any purpose other than disaster recovery, and a restore triggers re-application of pending erasures. That is a defensible position; having no position is not.

Verification, which is the part that gets audited

Per location: a deletion mechanism, a recorded outcome, and a sampled re-query afterwards asserting absence. Trusting the job's exit code is what produces a finding.

A dated design covering scope, mechanism per location, the backup position, the 30-day timeline with where the time goes, and how verification evidence is produced. If the current architecture cannot meet it — which is common — say so with the remediation plan and its duration, rather than agreeing to a date the architecture cannot support.