Fairness & Bias
Group and individual criteria, impossibility results, measurement under missing attributes, and mitigation costs.
11concepts
97flashcards
87minutes of reading
- 01 Discrimination in LLM-Mediated Decisions The correspondence-study design adapted to prompted models, what Anthropic's discrim-eval and the realistic-context follow-up measured, and why an anti-bias prompt that passes a clean evaluation fails once the prompt looks like production.
- 02 Disparate Impact and the Legal Frame How US anti-discrimination doctrine actually assigns liability, why the four-fifths rule is a screening heuristic rather than a standard, and the places where a passing fairness metric and a successful legal defence come apart.
- 03 Where Bias Enters the Pipeline The six distinct points at which disparity is introduced, why calling them all "biased data" prevents fixing them, and which stage each mitigation actually addresses.
- 04 Bias in Generative Models Why classification fairness metrics do not transfer to open-ended generation, the harm categories that appear instead, and the evaluation approaches that produce actionable findings.
- 05 Fairness Dynamics and Delayed Impact Why a constraint that equalises a decision has no built-in relationship to the outcome it is meant to improve, how deployed predictors reshape the distribution they are scored against, and what fairness transfer failures look like in real deployments.
- 06 Fairness of Exposure in Ranking Why a two-point difference in estimated relevance becomes a 37 percent difference in attention, how exposure constraints and stochastic rankings redistribute it, and what amortising fairness across many queries costs.
- 07 Group Fairness Criteria and Why They Conflict Demographic parity, equalised odds and calibration stated precisely, the impossibility result showing you cannot have all three, and what choosing between them commits you to.
- 08 Individual and Counterfactual Fairness Two attempts to define fairness for a person rather than a group, the Lipschitz condition that treats similar people similarly and the causal condition that asks what the decision would have been had the person's group been different, and why each relocates the value judgement into an object someone must build.
- 09 Measuring Fairness Without the Attribute The methods for estimating disparity when group membership is unavailable, why proxy inference introduces error in a direction that matters, and how to report a result built on an estimate.
- 10 Mitigation at Pre-, In- and Post-Processing The three points at which a fairness intervention can act, what each costs in accuracy and in flexibility, and the legal constraint that decides which are available.
- 11 Multicalibration and Subgroup Guarantees Why satisfying a fairness constraint on each protected attribute separately leaves intersections unprotected, what it means to ask for calibration on every computationally identifiable subgroup at once, and how a post-processing loop achieves it.