Human-in-the-Loop Design
Meaningful review rather than a rubber stamp, and designing against automation bias.
5 to work through
-
beginner Multiple choice
A claims system shows the reviewer a risk score of 0.87, the recommendation Reject, and a confirm button with the recommendation already selected. Median time on screen is 4 seconds. Audit records this as human oversight. What is the smallest change that makes the oversight real?
2 min answer -
advanced
A fraud system routes flagged transactions to human reviewers, who approve or reject within a target of 4 minutes each. A campaign triples flagged volume overnight. Reviewer headcount is unchanged. What happens to the quality of the human oversight, and what should the design do about it?
2 min answer -
advanced
A high-consequence decision requires human oversight of an automated recommendation. What makes that oversight meaningful rather than nominal?
2 min answer -
advanced Multiple choice
An anti-fraud system routes flagged transactions to human reviewers. Audit shows reviewers approved 98% of cases with a median handling time of 3 seconds. The control is documented as human oversight. What is the most accurate finding?
3 min answer -
advanced
An approval step has a 99.8% approval rate. What does that tell you and what would you change?
2 min answer
3 terms in this topic
Automation Bias
The well-documented tendency for people to defer to a system's output rather than assess it independently - which is why nominal human oversight prov…
patternHuman in the Loop Design
Placing human judgement at the points where automation should not decide alone — with attention to whether the human can actually exercise judgement.
metricReviewer Throughput Ceiling
The number of items a human reviewer can genuinely assess per hour, which bounds what any human-in-the-loop control can actually deliver regardless o…
Neighbouring topics
Assurance, Audit & Model Risk
General material on assurance, architectural governance and risk oversight.
Control Design vs Operation
A control that is well designed and never runs fails exactly like one that is absent.
Audit Evidence
Producing durable, tamper-evident proof as a by-product rather than as a project.
Certification Impact on Architecture
What SOC 2 and ISO 27001 actually require of a design, and what they do not.
Continuous Controls Monitoring
Testing controls continuously instead of sampling them once a year.
Segregation of Duties
Splitting authority so no single actor can both make and approve a change.
Change Advisory vs Automated Gates
Replacing a weekly board with evidence a machine produces on every change.
Risk Appetite
The stated tolerance that tells you which risks you are allowed to accept.
Risk Assessment Methods
Qualitative matrices, FAIR and scenario analysis, and the illusion of a precise score.
Security Design Review
Reviewing an architecture for security while changing it is still cheap.
Architecture Compliance Checks
Automating conformance to standards so review effort goes to the genuinely novel.
Exception & Waiver Management
Time-boxed, owned deviations with a remediation date, rather than permanent silence.
Design Authority
How an ARB should decide, what it should not review, and how it avoids becoming a queue.
Three Lines Model
Ownership, oversight and independent assurance, and where architecture sits in it.
Model Risk Management
Inventory, validation, monitoring and challenge for models that make consequential decisions.
AI Risk Tiering
Classifying a use case by potential harm, and the obligations each tier triggers.
Model Documentation
Model cards, intended use, limitations, and the record a regulator will ask for.
Model Evaluation & Red-Teaming
Adversarial testing of a probabilistic system with no fixed expected output.
Bias & Fairness Controls
Measuring disparate outcomes, choosing a fairness definition, and living with the trade-off.