A pipeline combines automated processing with human review at scale. How should the boundary between them be designed?

The routing decision

Route by model confidence and by consequence, not by a fixed sample. The automated path handles cases the model is confident about; humans handle uncertain cases and high-consequence cases regardless of confidence.

That requires calibrated confidence, which is a separate engineering problem: a model's raw output probability is frequently not a good confidence estimate, and using it as one sends the wrong cases to humans.

The design properties that determine throughput and quality

• Task design matters more than tooling. A review task that takes forty seconds instead of ninety doubles effective capacity, and most of that difference is in what is shown, in what order, and how the decision is captured.
• Quality control on the humans, through overlapping assignments on a sample, gold-standard items seeded into the stream, and agreement measurement. Human review is not automatically correct, and an unmeasured review process degrades.
• The feedback loop back to the model. Human decisions are training data, and a pipeline that does not capture them in a usable form is discarding its most valuable output.
• Queue prioritisation by urgency and consequence, so a time-sensitive item is not behind a backlog.
• Bounded queues with an escalation path, because a review queue that grows without bound eventually means items are processed too late to matter, which is a silent failure.

The failure mode that defines the domain

Automation ratchets in one direction. As the model improves, the cases reaching humans become systematically harder — the easy ones are automated away — so reviewer throughput falls and error rates rise even though the overall system improved.

Measuring reviewer performance without accounting for this penalises people for the system's success, and it is the most common analytical mistake in these pipelines.

The architectural consequence

The human step is a service with a latency distribution, a capacity limit and a failure mode, and it should be modelled that way rather than as an escape hatch. That means an SLA, monitoring, capacity planning, and a defined behaviour when the queue exceeds what the available reviewers can process — usually raising the automation threshold and accepting a measured quality reduction, decided in advance rather than during a backlog.