Fairness Through Unawareness

A credit team deletes the ethnicity column from the training data and reports to the risk committee that the model can no longer discriminate. It is a natural claim and an anti-pattern: a model with a hundred features does not need a column labelled ethnicity, because the information is redundantly encoded across the features that remain. Postcode carries residential segregation. Employer, education, device type, name-derived features and transaction patterns each carry a fraction. A model optimising for repayment will use whatever combination predicts, and that combination can track the attribute closely.

The test takes an afternoon: train a classifier to predict the protected attribute from the remaining features. On consumer data it typically lands well above the 50% a coin would give on two groups, which means the pathway is intact.

Why it matters

Unawareness does one real thing: it removes the attribute as an explicit input, which matters where using it in a decision is unlawful. It provides no assurance about outcomes, and that is what a regulator, a customer and a journalist ask about.

The second-order damage is worse than the first. Teams that delete the attribute usually delete it everywhere — from the warehouse as well as the feature store — and then cannot measure the outcome gap at all. The organisation has converted a measurable risk into an unmeasurable one and reported it as a control improvement. A regulator treats the absence of measurement as the finding.

Implementation patterns

The correct architecture separates the two uses of the attribute:

• Out of the decision path. The protected attribute is not a feature, not in the serving payload, and contract tests fail if the column appears in a training set.
• Retained for measurement, in a separate store with its own access control, purpose limitation, logging and retention, keyed to the decision record by a one-way identifier.
• Inferred where collection is barred. Where the law prevents collecting it, use a documented probabilistic proxy. The CFPB's 2014 white paper describes Bayesian Improved Surname Geocoding, which combines the composition associated with a surname and with the applicant's census geography, and publishes the method.
• Aggregate only. Report group approval rates, score distributions and error rates with confidence intervals; never attach an inferred label to an individual.
• Choose the fairness definition explicitly and record who chose it, because parity of approval rates and parity of error rates cannot both hold when base rates differ.
• Re-measure after every retrain. Proxy relationships shift when the data does.

Industry example

Fair-lending supervision is where this has been worked out in public. US non-mortgage lenders generally do not collect applicants' race, so the regulator built a proxy method rather than accepting that the question was unanswerable, and published it in 2014 with the code. The lesson for architecture is the separation it implies: the proxy exists purely for population-level monitoring and never re-enters the decision.

Failure scenarios

• Silent proxy discrimination. Approval rates diverge by 7 points and the model has no protected attribute anywhere in it.
• The measurement hole. The attribute was deleted everywhere; nobody can answer the regulator and the remediation starts with a year of data collection.
• Proxy leakage. A helpful engineer joins the proxy table to the feature store to improve accuracy, turning a monitoring dataset into a discriminatory input.
• Underpowered dashboards. A monthly gap computed on 300 decisions is noise reported as assurance; detecting a 5-point gap near a 50% base rate needs roughly 1,600 decisions per group.
• Definition drift. The dashboard silently measures parity while the legal exposure is about error rates.

Trade-offs

Keeping the attribute for measurement means holding sensitive data, which costs access control, retention discipline and a breach consequence. Inferring a proxy avoids collection and adds measurement error that biases the estimated gap, usually towards zero. Not measuring costs nothing until a supervisor asks, at which point it costs the most of the three. The defensible middle is a small purpose-limited measurement store, or a documented proxy, with aggregate reporting and a hard architectural break between it and the serving path.

When not to use it

Removing the attribute from the features is correct and sufficient when the law forbids its use in the decision — most regulated lenders do exactly that. The mistake is only in what is claimed for it. Unawareness is also a reasonable whole position in the narrow case of a model whose handful of inputs genuinely carry no group information, such as a few clinical measurements, where there is no plausible proxy channel. Even there, the outcome monitoring is the part doing the work, and if monitoring is impossible because the volume is tiny, say that plainly rather than substituting a claim the data cannot support.

Interview question

Q: A team tells you their model is fair because it does not see the protected attribute. How do you test that claim in a week, and what would you change regardless of the result?

What a strong answer covers: train a classifier to recover the attribute from the remaining features as the direct test · measure outcome gaps by group with confidence intervals and state the power · separate measurement storage from the serving path with enforced controls · choose and record a fairness definition · and the change to make either way, which is that the fairness claim must be about outcomes rather than about inputs.

Quick check

Quiz: A model has no protected attribute among its 120 features and its approval rates differ by 7 points across groups. Is it discriminating? — Possibly: unawareness says nothing about outcomes, and correlated features can reproduce the attribute; the gap is the thing to investigate.

Flashcard: What is the second, worse cost of deleting a protected attribute everywhere? — Losing the ability to measure the outcome gap, which converts a measurable risk into an unmeasurable one.