Tail Latency Amplification
The effect where a request that fans out to many services experiences the worst case of all of them, making rare slowness common at the user level.
The arithmetic that explains why large systems are slow at the tail even when every component looks healthy.
If a service responds within 10 ms for 99% of calls, and a user request fans out to 100 such services in parallel and must wait for all of them, the probability that all 100 are fast is 0.99^100, about 37%. So roughly 63% of user requests hit at least one slow component. A one-in-a-hundred event at the component level is the common case at the user level.
This is why teams optimising averages see no improvement in user experience: the average was never the problem. It is also why p99 at the service level is the number that matters for a fan-out architecture, and why p99.9 matters for the services underneath it.
The mitigations, each attacking a different part. Reduce fan-out — fewer parallel dependencies is the most effective and least used lever. Hedged requests cut the tail by racing a second replica. Return partial results where the product allows it, rendering without the recommendations rather than waiting. Tighten per-dependency deadlines so a slow component is abandoned rather than waited for. And attack the sources of tail latency directly: garbage collection pauses, cold caches, noisy neighbours, and queueing at high utilisation.