A platform uses AI to propose substitutions when items are unavailable. Where should the human sit in the loop, and what determines it?

What determines the placement

The cost of being wrong, and whether the error is detectable and correctable.

• Low cost, correctable — the system acts and the human can undo. Substituting a brand of milk.
• Moderate cost — the system proposes and the human confirms, with a good default so confirmation is cheap. Substituting across a category.
• High cost or irreversible — the human decides, with the system providing information. Substituting an item where the customer has a stated allergy or dietary requirement.

The mistake in both directions is uniform treatment: requiring confirmation for everything makes the feature useless, and acting autonomously on everything produces the errors that destroy trust.

The design

1. Confidence-based routing. The system acts autonomously above a threshold, proposes below it, and escalates below a lower one. The thresholds are product decisions, adjustable without deployment, and set from measured error rates rather than intuition.

2. Which human, and when. For substitutions there are two candidates with very different properties: the shopper, present and able to see the shelf but under time pressure; and the customer, who knows their preference and may not respond in time. The design must handle non-response with a defined default rather than stalling.

3. Preferences captured in advance, which converts many decisions into pre-approved policy — "always substitute within brand", "never substitute this category" — and removes the interaction entirely for those cases.

4. The proposal must be cheap to evaluate. A confirmation requiring a human to read a paragraph will be approved without reading. Show the original, the proposal, the difference and the price change, and make approval one action.

5. Feedback captured. Every acceptance, rejection and correction is training signal and evaluation data, and capturing it is what makes the system improve rather than plateau.

What must not be delegated

Anything where the failure is a safety or regulatory matter. Dietary restrictions, age-restricted items, prescription-related products. These need hard rules rather than model confidence, because a confidence threshold is a probability and the requirement is categorical.

The measurement

Acceptance rate, correction rate, and downstream outcome — refunds, complaints, repeat purchase. A high acceptance rate with high complaint rate means humans are approving without evaluating, which is the failure mode a confirmation step is meant to prevent and frequently does not.