All outbound traffic from a cluster passes through an egress proxy that enforces allowlists and TLS inspection. The proxy does not fail; it gets slow, with p99 rising from 20 ms to 6 s. What happens across the platform over the next ten minutes?

Minute by minute

Minutes 0 to 2. Calls to external providers, payment authorisation, email, third-party APIs, start taking seconds. Callers with generous client timeouts wait. Each waiting call holds a thread or a connection from a bounded pool.

Minutes 2 to 5. Pools fill. A service whose pool is exhausted stops serving requests that have nothing to do with the external call, because the pool is shared. This is the amplification step: a dependency used by 10% of requests takes down 100% of the service. Health checks that only test the process still pass, so the load balancer keeps sending traffic.

Minutes 5 to 10. Client timeouts fire and retries begin. Retries go through the same proxy, adding load to the thing that is already slow. Offered load rises while capacity falls, which is the shape of every congestion collapse. Upstream services waiting on these services exhaust their own pools, and the failure walks up the call graph.

Where it amplifies

Three multipliers, in order of severity: shared connection pools turn a partial dependency into a total outage; retries without a budget convert a slowdown into a load increase; and TLS inspection, which makes the proxy do real cryptographic work per connection, means it degrades non-linearly once CPU-bound rather than queuing gracefully.

What the user sees

Not an error page. Slow pages, then timeouts, on features that have no external dependency at all. The status page is green because every service is up. This is the archetype of a gray failure: nothing is down, everything is broken.

What stops it

1. Bulkheads. A separate, small connection pool for external calls, sized so exhausting it cannot starve local work. This is the single change that bounds the blast radius, and it is the one most often missing.
2. Timeouts shorter than the caller's remaining budget, so slow becomes an error before it becomes a thread leak. A 30-second default in an HTTP client is the usual culprit.
3. A circuit breaker that trips on slow-call rate, not just on errors. A breaker configured for error rate never opens here, because there are no errors: there are slow successes.
4. Retry budgets, capping retries as a fraction of total requests, typically 10%, so retries cannot become the load.
5. Load shedding at the proxy, rejecting fast when its queue exceeds a threshold. A fast rejection is a gift to the caller; a slow success is not.

What would have to be true for it to self-heal

Offered load would have to fall as fast as capacity did. With retries in the picture it rises instead, so the system cannot self-heal without a mechanism that removes load: shedding at the proxy, breakers at the callers, or an operator turning something off. Autoscaling the proxy usually makes it worse if the constraint is a downstream rate limit or a licence-bound inspection capacity.

What to watch

Queue depth and p99 at the proxy, and pool utilisation per service as a first-class metric. Pool saturation is the mechanism by which a partial dependency becomes total, and almost nobody alerts on it until after the first incident.

When this is still the right design

When outbound traffic must be controlled and evidenced - allowlists, data loss inspection, a regulator asking what left the network - a chokepoint is the only practical answer, and the alternative of per-service egress rules does not survive contact with 300 services in production. Choose the chokepoint, and accept that its cost is a shared dependency you must engineer around. Build the chokepoint, then engineer for its slowness rather than for its failure, because slow is the failure mode that actually happens.