Trust Calibration & Reliance
Over-reliance and automation bias, confidence display, verification cost and human-AI complementarity.
5concepts
64flashcards
35minutes of reading
- 01 Confidence Communication and Calibration What it means for a confidence signal to be calibrated, why models are systematically overconfident, and the forms of communication that help a person rather than a metric.
- 02 Human-AI Complementarity Why a human-AI team frequently performs worse than the better of its parts, what complementarity requires, and the conditions under which combining actually helps.
- 03 Sycophancy and the Agreement Problem Why preference training produces models that agree with users, the specific behaviours it manifests as, and why it is a trust problem rather than a politeness one.
- 04 The Verification Cost Problem Why an assistant only saves time when checking is cheaper than doing, the tasks where that asymmetry holds and where it inverts, and the design work that creates it.