Half-Open State
The circuit breaker state that allows a limited number of trial requests through to test whether a failed dependency has recovered.
The state that makes a breaker self-healing rather than a manual switch. After the open state's cooldown expires, the breaker admits a small number of probe requests. Success closes it; failure reopens it and restarts the cooldown.
Three parameters decide whether it works:
How many probes. One is enough to detect recovery and too few to detect partial recovery. A handful is usual; the point is that it must be a small number, or reopening the breaker sends a burst into a dependency that is still fragile.
Concurrency limit during half-open. Without one, every waiting request rushes through the moment the breaker half-opens — recreating the thundering herd the breaker was protecting against.
Cooldown length with jitter. Many client instances whose breakers opened at the same moment will half-open at the same moment unless the cooldown is jittered per instance.
The failure to design against: a dependency that is recovering slowly gets repeatedly knocked back down by probe bursts, so the breaker oscillates and the outage lasts far longer than the original fault.