Human-AI Complementarity
Why a human-AI team frequently performs worse than the better of its parts, what complementarity requires, and the conditions under which combining actually helps.
The assumption behind every human-in-the-loop design is that the combination beats either alone. Studies across domains repeatedly find otherwise: the team often performs at or below the level of the better member, and sometimes below both. Understanding why is what separates a design that helps from one that adds a step.
Why combining can fail
The human cannot tell when to override. Complementarity requires the person to accept correct outputs and reject incorrect ones, which requires distinguishing them. If they could reliably do that, they could do the task. Without a signal that separates the cases, overrides are approximately random and add noise.
Anchoring. Seeing the model's answer first changes the human's judgement toward it, so the independent perspective that was supposed to add value is compromised before it is formed. Showing the model's output after the human commits preserves independence and costs time.
Selective overriding is worse than none. A person who overrides only when they disagree is applying their judgement precisely where their model of the system is weakest, which can be worse than a policy of always accepting or always rejecting.
Effort withdrawal. Given an assistant, people invest less effort in the task, so their independent contribution drops. The combination is then not "human plus model" but "model plus a lightly checked human".
What complementarity requires
The combination helps when the two make different errors and something identifies which regime applies.
Different information. The human sees context the model does not: the patient in front of them, the customer's history, the thing that happened last week. Where the human's advantage is informational rather than analytical, the design should route on that: the model handles cases where its inputs are complete, and the human handles cases where context matters.
Different failure modes. A model that fails on rare cases and a human who fails on volume are complementary if the routing sends rare cases to the human.
A reliable routing signal. Complementarity is realised by allocation, not by review. Deciding in advance which cases go where, based on a property that predicts who does better, captures the gain that unstructured review does not.
When it breaks
The measurement is usually missing. Most deployments never compare the joint system against the model alone, so a design that adds a human step is assumed to improve accuracy without evidence. That comparison is cheap and is the one that should gate the design.
Expertise changes the answer. A domain expert and a novice have different complementarity profiles with the same model, so a design that works for one can fail for the other, and evaluation on one population does not transfer.
Skill atrophies. Sustained reliance degrades the human capability the design depends on, so a team that was complementary at deployment may not be two years later. This is documented in aviation and medicine and there is no reason to expect it to be different here.
Accountability without capability is the worst arrangement. Placing a human in the loop so that someone is responsible, without giving them the information, time or authority to add value, creates liability transfer dressed as a safeguard. It is common and it is worth naming.
14 flashcards for this concept
Click a card to reveal the answer.