A product analytics team needs per-user aggregation over two years of events without holding data that re-identifies people if the warehouse leaks. Which approach actually delivers that?
Show the full answer Hide the answer
The deciding property
The requirement has two halves that pull against each other: stable linkage across two years and no re-identification from the leaked dataset alone. Any scheme where the mapping can be recomputed from the leaked data satisfies the first and fails the second.
Why the token vault is the answer
A random token carries no information about the subject, so the only path back is the mapping table, and that table lives in a different system with a different access path. The leaked warehouse is then genuinely a set of behavioural rows against opaque ids. Revocation and rotation are possible because the mapping is data you own rather than a function of the input.
What it costs: a lookup on the ingest path, which you batch rather than call per event; a new tier-1 dependency, because if the vault is down you cannot tokenise or re-identify; and a concentrated target — the vault is now the thing an attacker wants, so it gets its own access model, its own audit and its own key rotation.
Why the other options fail
- SHA-256 of the email. Deterministic hashing of a low-entropy identifier is reversible by enumeration. Commodity hardware runs billions of SHA-256 evaluations per second, and the space of plausible addresses is far smaller than the hash space, so a leaked column of hashes is a leaked column of emails. A keyed hash (HMAC with a separately held key) is a real improvement, and it is the right cheap answer when you never need to reverse the mapping — but rotating the key breaks every historical join, which is exactly what a two-year analysis needs.
- Encrypt with the key stored beside the data. This is encoding, not protection. One breach takes both. It is common because it passes a checklist that asks "is the column encrypted?".
- Drop the identifier and aggregate by session. This is correct when longitudinal analysis is not required, and it is the cheapest privacy control available. It fails the stated requirement, because per-user aggregation over two years is the thing being asked for. If the analytics team cannot justify that requirement, prefer this option and delete the problem.
What flips the decision, and when not to tokenise at all
| If this changes | Choose | Because |
|---|---|---|
| Re-identification is never needed | HMAC with a held key or no identifier at all | No mapping to protect or operate |
| Data is published outside the company | k-anonymity or aggregation thresholds | Tokens still permit linkage attacks across releases |
| Events arrive at very high rate with tight latency | Batch tokenisation offline | A vault lookup per event puts a tier-1 dependency on the hot path |
Pseudonymised data remains personal data. The gain is a reduced consequence if it leaks, not an exemption from the rules.