PII Detection and Redaction Pipelines
Why regular expressions catch the easy half, how the recall-utility tradeoff differs between training corpora and live traffic, and the re-identification risk that survives field-level redaction.
Removing personal information from text is the first control most teams reach for and the one whose limits are least examined. It is genuinely useful, it is never complete, and treating its output as clean is how a pipeline acquires a false assurance that shapes every decision downstream.
What each layer catches
Pattern matching handles structured identifiers: email addresses, phone numbers, card numbers with checksum validation, national identifiers with known formats. High precision when the format is genuinely constrained, and it catches nothing unstructured. It is the cheapest layer and the one whose coverage is easiest to overestimate.
Named entity recognition finds people, organisations, locations and dates. Recall on well-formed text is good and degrades on the informal, misspelled, code-switched text that real user input consists of. It also has no notion of whether an entity is sensitive in context: a public figure's name in a news article and a patient's name in a note are the same entity type.
Contextual classification, increasingly with a language model, judges whether a span is sensitive given its surroundings. It handles the cases the first two miss, including free-text disclosures, and it costs a model call per document and produces errors that are harder to characterise than a regex's.
Structural exclusion is the layer people forget: not collecting the field, dropping the column, or hashing the identifier at ingestion. It is the only layer with no recall problem, and it is available far more often than it is used.
Two settings, two operating points
For a training corpus, recall matters far more than precision. A false positive removes a small amount of data from a large corpus at negligible cost; a false negative places a personal identifier in a model's weights permanently, where it may become extractable. The correct operating point is aggressive, and over-redaction is a price worth paying.
For live traffic, over-redaction destroys the utility of the request. Redacting a customer's own name from their support message means the assistant cannot address them. The correct operating point is different, and often the right answer is not redaction at all but scoping: process the data in place with access controls rather than sanitising it for wider distribution.
Conflating the two settings, and applying a training-corpus threshold to production traffic, is a common source of a feature that mysteriously degraded.
When it breaks
Redaction does not prevent re-identification. Removing names and identifiers leaves quasi-identifiers, and the combination of postcode, date of birth and sex has long been known to identify a large share of a population. Field-level redaction addresses direct identifiers and does nothing about the joint distribution, which is what \(k\)-anonymity and its successors exist to address and what a redaction pipeline does not.
Placeholders leak structure. Replacing every name with [PERSON] preserves how many people were mentioned and where, and consistent per-document pseudonyms preserve the co-occurrence graph. Whether that matters depends on the threat model, and it is rarely considered.
Evaluation requires labelled data that is itself sensitive. Measuring recall means having a corpus with known personal information, which is the corpus you cannot freely share or annotate. Most redaction pipelines are deployed with a precision estimate and a recall guess.
The pipeline becomes an assurance it cannot support. Once a dataset is labelled redacted, downstream teams treat it as safe and relax controls. Since recall is never complete, the correct posture is that redaction reduces exposure rather than eliminating it, and access controls should remain proportionate to what the data was before redaction.
12 flashcards for this concept
Click a card to reveal the answer.