A consumer fintech accumulates transaction, behavioural and derived data. How should retention, archival, deletion and regulatory holds be designed so they do not conflict?
Show the full answer Hide the answer
The conflict to resolve first
Regulation requires you to keep some data and to delete other data, and the same record can be subject to both. A transaction record may be subject to a seven-year financial retention obligation while the personal data attached to it is subject to an erasure right. Designing these as one policy guarantees violating one of them.
The model that resolves it
Separate the record from the identity. Keep transaction facts under the retention obligation with a pseudonymous subject reference, and keep the mapping from reference to person in a separate store governed by the erasure rules. Deleting the mapping renders the retained records non-identifying while preserving them for their regulatory purpose.
This has to be designed early. Retrofitting it means rewriting every table that embedded an email address or a phone number, which is nearly every table.
The lifecycle stages to design explicitly
| Stage | Storage | Access pattern | Governing rule |
|---|---|---|---|
| Hot | Operational store | Interactive, frequent | Product requirement |
| Warm | Analytical store | Batch, occasional | Analytical need |
| Cold | Object storage, compressed | Rare, restore-first | Regulatory retention |
| Deleted | Gone from all of the above, including backups | — | Erasure obligation |
The stage that breaks in practice is the last one. Deletion that does not reach backups, replicas, caches, analytics copies, search indexes and third-party processors is not deletion, and every derived copy is a place the obligation is silently unmet.
Legal hold
A hold must override deletion, including scheduled and user-requested deletion, and it must be recorded with its scope, its reason and its author. That means deletion cannot be a hard delete executed by a cron job; it must go through a service that checks for holds and produces evidence of what it did.
The design consequences
- Data must be inventoried and classified to be governed at all. You cannot apply a retention policy to data whose existence is unknown, and derived copies are where the unknowns live.
- Deletion must be propagated as an event to every derived store, and its completion must be tracked rather than assumed.
- Retention should be enforced by the platform — table-level policies, object lifecycle rules — rather than by application code that someone must remember to write.
- Backups need their own policy. An immutable backup taken before a deletion request still contains the data, and the usual resolution is a documented backup retention window after which it ages out, agreed with legal rather than assumed by engineering.