Fan-Out Latency Amplification
The effect by which a request that depends on many parallel sub-requests is governed by the slowest of them, so rare slowness becomes common at the user level.
If a request fans out to 100 services in parallel and waits for all of them, the response time is the maximum, not the average.
With each service having a 1% chance of exceeding 100 ms, the probability that at least one does is 1 − 0.99¹⁰⁰ ≈ 63%. A tail event that affects one request in a hundred at the component level affects most requests at the user level.
This is why large fan-out architectures — search, recommendation, feed assembly — are dominated by tail latency rather than mean latency, and why their operators invest disproportionately in p99 and p999.
Mitigations, in the order they are usually applied:
Hedging — issue a duplicate request to another replica after a short delay and take the first response. Cuts the tail dramatically at a small percentage increase in load.
Reduce fan-out — batch, cache, or restructure so fewer parallel calls are needed. The most durable fix.
Return partial results with a deadline, rather than waiting for the slowest component. Suits ranking and recommendation, where a slightly worse answer now beats a better answer later.
Attack the causes of the tail — garbage collection pauses, cold caches, lock contention, noisy neighbours, and retries.