A dependency degrades to 2-second latency rather than failing outright, and retries triple the load on it. Why is slow worse than down, and which mechanisms actually prevent the amplification?

Why slow is worse than down

A dependency that fails fast returns an error in microseconds. The caller's thread is released, the circuit breaker opens, the fallback runs, and the caller's capacity is barely touched.

A dependency that slows to 2 seconds holds every caller's connection, thread, or coroutine for 2 seconds. By Little's law, concurrency = arrival rate × latency, so a 20× latency increase requires 20× the concurrency to sustain the same throughput. The caller exhausts its connection pool, its own latency rises, and the caller now looks slow to its callers — the failure propagates upward as saturation rather than as errors.

Then retries arrive. A client retrying twice against a saturated dependency triples its load precisely when it is least able to serve it, which is the amplification. And this is the standard, normally-correct retry behaviour doing exactly what it was designed to do.

The mechanisms that actually help

• Deadline propagation. The originating request's deadline travels through the whole call chain, and each hop passes the remaining budget. A service that sees 40 ms left does not start a 2-second call — it fails immediately. This single mechanism eliminates most of the wasted work, because the majority of the load on a saturated dependency is requests whose callers have already given up.
• Retry budgets, not per-call retry counts. A budget caps retries as a fraction of total requests — often a few percent. Under widespread failure the budget is exhausted and retries stop, converting an amplifier into a fixed ceiling. Per-call limits do the opposite: the worse things get, the more retries happen.
• Circuit breakers keyed on latency, not only on errors. Breakers that trip only on exceptions never open in this scenario, because the dependency is returning correct answers, slowly. The breaker must consider latency and concurrency.
• Adaptive concurrency limits. Measure achievable throughput and shrink the in-flight limit when latency rises, so the caller sheds load rather than queueing it. Queueing in front of a saturated service converts a capacity problem into a latency problem for everyone.
• Hedged requests with cancellation — a second attempt after the p95, with the loser cancelled. This improves tail latency and is safe only with cancellation and a strict budget; without them it is a deliberate load multiplier.
• Bounded pools with fail-fast on exhaustion. When the pool is empty, reject immediately rather than queueing indefinitely.

The diagnostic signature

Latency rising while error rate stays flat, then errors appearing at the caller before the dependency. Saturation propagates upward, so the alert usually fires on the wrong service — which is why the metric to watch is concurrency and queue depth per dependency, not just its latency.