After adding circuit breakers, a partial outage now lasts three times longer than it used to. What is likely happening?

The likely mechanism: breaker oscillation

The dependency is recovering slowly — it can serve some traffic but not full load. The breaker opens, waits out its cooldown, half-opens, and every client that has been waiting rushes through at once. The dependency, which was managing a trickle, is knocked back down. The breaker reopens. Repeat.

Each cycle delivers a burst that prevents recovery, so the outage persists far longer than the original fault would have caused.

The specific misconfigurations

No concurrency limit in half-open. This is the main one. Half-open should admit a small, capped number of probes, not "everything that was blocked".

Unjittered cooldown across instances. Fifty client instances whose breakers opened within the same second will all half-open within the same second. Jitter the cooldown per instance.

Too many probe requests, so even a capped half-open sends a meaningful burst.

All-or-nothing transition. A breaker that goes straight from half-open to fully closed sends 100% of traffic the moment one probe succeeds. Ramping — 10%, 25%, 50%, 100% with health checks between — lets the dependency recover under gradually increasing load.

What else to check

Are slow responses counted as failures? If not, the breaker never opens on the actual symptom and threads block anyway.

Is the fallback cheap? A fallback that itself calls something expensive can make the open state worse than the closed one.

Are breakers stacked? A breaker in the mesh and another in the client library, with different thresholds, produce behaviour neither team predicts.

What a strong answer adds

The general principle: recovery is a distinct design problem from failure detection, and it is usually the one that is skipped. Anything that resumes traffic after an outage — breakers, autoscalers, retry queues, cache refills — needs an explicit ramp, or the resumption becomes the second incident.