Group Fairness Criteria and Why They Conflict

There is no metric for fairness. There are several, they encode different ethical positions, and a theorem says you cannot satisfy them simultaneously except in degenerate cases. This is not a gap awaiting research. It is a structural fact that turns "make the model fair" into a question about which unfairness is acceptable.

The three families

Write \(A\) for the protected attribute, \(Y\) for the true outcome, and \(\hat{Y}\) or \(R\) for the prediction or score.

Independence, also called demographic parity or statistical parity, requires \(\hat{Y} \perp A\): the selection rate is equal across groups. It says nothing about accuracy and can be satisfied by selecting randomly within a group. It is the right criterion where the outcome variable itself is suspect, for instance where historical labels encode past discrimination, and it is wrong where genuine base rates differ for legitimate reasons.

Separation, of which equalised odds is the canonical form, requires \(\hat{Y} \perp A \mid Y\): equal true positive and false positive rates across groups. It conditions on the truth, so it permits different selection rates where base rates genuinely differ, and it demands that errors be distributed equally. Equal opportunity is the relaxation requiring only equal true positive rates, appropriate where a false negative is the harm that matters.

Sufficiency, of which calibration by group is the canonical form, requires \(Y \perp A \mid R\): among everyone assigned score \(s\), the outcome rate is the same regardless of group. A score of 0.7 means the same thing for everyone. This is the criterion that matches what a decision-maker consuming a score needs, and it is the one risk-scoring systems are usually designed for.

The impossibility

When base rates differ across groups and the classifier is not perfect, calibration and equalised odds cannot both hold (Kleinberg et al., 2017, arXiv:1609.05807; Chouldechova, 2017, arXiv:1703.00056).

The intuition is direct. If group A has a higher base rate and the score is calibrated, then at any threshold a larger share of group A's positives are correctly identified, so the error rates cannot match. Forcing equal error rates breaks the score's meaning across groups. Any two of the three families are jointly achievable only under conditions, equal base rates or a perfect classifier, that do not obtain in practice.

This is precisely the disagreement at the centre of the COMPAS recidivism debate: the vendor argued the tool was calibrated by group, and the critics showed unequal false positive rates. Both were correct, and the disagreement was about which criterion should govern, not about the numbers.

Choosing

The choice depends on what the decision does and what the errors cost whom. Where a false positive imposes a serious burden on an individual, equalised false positive rates matter and calibration is secondary. Where a score is consumed by a downstream decision-maker who will act on its stated meaning, calibration matters. Where the labels themselves reflect historical discrimination, conditioning on them at all is questionable and demographic parity is the more defensible target.

Making that choice explicitly, and documenting the reasoning, is the substantive act. Selecting a metric because a library implements it is the common alternative.

When it breaks

Group fairness says nothing about individuals. All three criteria are satisfiable while treating similar individuals within a group inconsistently. Individual fairness, requiring similar treatment for similar people, needs a similarity metric that encodes the value judgement it was meant to avoid.

Intersectionality multiplies the problem. Parity on race and on sex separately does not imply parity on their intersection, and subgroup sample sizes shrink until the estimates are too noisy to act on. Reporting on marginal attributes only is the standard practice and it systematically hides intersectional harm.

Fairness through unawareness does not work. Removing the protected attribute leaves proxies: postcode, name, purchase history, device. The model reconstructs the attribute and the practitioner loses the ability to measure disparity, which makes the intervention worse than useless.

The metric is not the goal. Satisfying a criterion is compatible with a system that should not exist. Fairness metrics evaluate how a decision is distributed; they cannot evaluate whether the decision should be automated at all.