A travel search fans out to 40 suppliers with latencies from 80 ms to 4 seconds and varying reliability. Which timeout strategy should the aggregator use?

Why the global deadline comes first

The user is waiting for a page, not for a supplier. The only latency that matters is the one the user experiences, so the budget must be owned at the level where it is felt and then allocated downward.

Per-supplier timeouts with no global budget produce the classic aggregation failure: each supplier is individually within its timeout, and the page still takes eight seconds because the timeouts were never summed. Nobody is at fault and the experience is bad.

Why partial results are the normal case

With 40 suppliers, the probability that all 40 respond quickly is low even when every supplier is healthy. Designing for completeness means designing for the tail of a 40-way fan-out, which is dominated by whichever supplier is having its worst moment.

So the aggregator must treat "returned some results by the deadline" as success, not as degradation. That has product consequences worth stating out loud: results are non-deterministic between identical searches, and the UI must not imply exhaustiveness.

Why the other options fail

Uniform timeout at the slowest supplier's p99. Sets everyone's latency to the worst participant. It also wastes the fast suppliers' headroom — there is no reason to wait 4 seconds for a supplier that normally answers in 80 ms, because at 400 ms it is already anomalous.

No timeout. The aggregator's latency becomes the maximum over 40 suppliers, and one hung supplier holds resources indefinitely. This is how thread pool exhaustion propagates a single supplier's outage into a full search outage.

Per-supplier p99 with no global deadline. Better, but still unbounded in aggregate, and it has a subtle flaw: a supplier's own p99 is the wrong reference point. The right question is not "how long does this supplier usually take?" but "how long is this supplier's answer still worth waiting for?"

The complete design

• Global deadline set from the product requirement, say 2 seconds at p95.
• Per-supplier timeout = min(supplier's expected latency ceiling, remaining global budget).
• Speculative retry to a fast supplier only if budget remains — hedging is affordable when there is slack and forbidden when there is not.
• Circuit breakers per supplier, so a persistently failing one is skipped rather than waited on.
• Cached last-known results as the fallback for suppliers that miss the deadline, clearly marked as indicative and always re-validated at booking.
• Attribution in telemetry: which suppliers made the deadline, per search. Without it, supplier reliability is invisible and unmanageable commercially.

The generalisable rule

Timeouts are not a property of a dependency; they are an allocation of a budget owned by the caller who faces the user. Any system where each hop sets its own timeout independently has no latency guarantee at all, however carefully each individual value was chosen.