Automation Bias
The tendency of a human reviewer to accept a system's output rather than evaluate it, which is what turns human oversight into a rubber stamp.
Human oversight is frequently the control relied upon to justify deploying an automated decision system: the model recommends, a person decides. The control is weaker than it appears, and predictably so.
People accept algorithmic recommendations at very high rates, particularly under time pressure, when the system is usually right, when the reviewer lacks the information to disagree, and when disagreeing carries more personal risk than agreeing. A reviewer processing sixty cases an hour is not evaluating sixty decisions.
Design choices either mitigate this or worsen it. Presenting the evidence rather than the conclusion — showing what the model observed instead of leading with its answer — produces genuine evaluation. Removing the throughput target that makes scrutiny impossible. Requiring a recorded reason when overriding and when agreeing on high-consequence cases. Measuring the override rate: a rate near zero is evidence the control is not operating, not evidence the model is excellent. Calibrating the reviewer with periodic known-answer cases, which both maintains skill and measures whether the review is real.
For a regulator, "a human reviews it" is increasingly not sufficient; the question is whether the review is meaningful, and the evidence for that is the override rate and the reasoning recorded alongside it.