A consumer app logs full request and response bodies for 30 days to chase a client bug. The bodies contain home addresses and partial payment identifiers. Which change most reduces foreseeable harm?
Show the full answer Hide the answer
The deciding property
Rank privacy controls by how many durable copies of the data they remove, not by how serious they sound.
A log event does not live in one place. A typical pipeline fans a single event into three to six durable copies: the agent's local spool, the aggregation buffer, the hot search index, cold object storage, a cross-region replica, and the dead-letter queue that holds whatever failed to parse. Add the copies nobody counts: cached dashboard results, a CSV exported to a laptop during an incident, and the vendor's own retention, governed by their contract rather than yours.
Capture-time minimisation is the only control that removes the data from every one of those copies at once, because the field never enters the pipeline. Everything else operates on a subset of the copies, after the data has already crossed the network.
The erasure obligation makes the same point from the legal side. Under the GDPR regime in force since 2018, an erasure request covers personal data wherever it is held, including logs. A field you never captured needs no erasure mechanism; a field in six copies needs six, one of which is a dead-letter queue nobody has an index for.
What it costs
Diagnostic power, and it is a real cost rather than a formality. An allowlist means the next unanticipated bug is harder to debug, because the field that would have explained it is not there. Pay for it deliberately: capture the request's shape and outcome rather than its content — field presence, lengths, types, validation error codes, a hash for correlation — plus a consented time-boxed capture path for one user when a real payload is genuinely needed.
The decision rule
If a field would be a problem in a breach, it should not be in a log. If you need it to operate, capture a derived form of it: a hash for correlation, a bucket for analysis, a last-four for support. And the test to put into design review, in one line: name every durable copy this field will land in, and who can read each one. Teams that cannot complete that sentence do not know their own blast radius, and the architect is usually the only person in the room who can.
When not to minimise
If the payloads genuinely carry no personal data — an internal batch job over reference data, a build system's artefacts — full-body capture is the cheapest debugging tool available and stripping it buys nothing. The control follows the data, not the habit.
Why the other options fail
Cut retention to 7 days. A sound second move: exposure scales with the window, so this cuts it roughly four-fold. It does nothing for a breach on day two, and the data still reaches the index, the replicas and the backups, whose retention is often set independently and quietly longer than the log store's.
Encrypt at rest and restrict to on-call. Wrong threat model. Encryption at rest defends against someone taking the disk or the storage bucket. The realistic exposure here is a legitimate query by one of the people who must have access, an export to a laptop, or an over-broad grant that nobody revokes. The data remains readable by design to exactly the population that is large enough to matter.
Access audit reviewed monthly. Detection, not prevention, and slow detection at that: a monthly review gives a median lag around two weeks. Audits are worth having, and they report the incident after the data has been read.
Regex redaction at index time. The most common choice and the most deceptive. The data has already travelled the network and sits in spools, buffers and the dead-letter queue, which the indexer never touches. Regexes also need a shape to match, and a home address has no shape; every format you did not anticipate passes through, and each miss is permanent.