Jitter

Exponential backoff alone solves the wrong half of the problem. If a thousand clients fail at the same instant — which is what a dependency outage produces — they all wait 1 second, all retry simultaneously, all fail, all wait 2 seconds, and retry simultaneously again. The dependency is hit by synchronised waves that are perfectly designed to prevent recovery.

Jitter breaks the synchronisation by randomising each client's delay. Full jitter — choosing uniformly between zero and the current backoff ceiling — is the variant that performs best in published analyses and is the sensible default. It spreads the retries into a continuous, absorbable trickle rather than a series of spikes.

The same reasoning applies well beyond retries, and this is where the concept earns its keep: any periodic behaviour that is synchronised across a fleet becomes a spike. Cron jobs on the hour, cache entries with identical TTLs expiring together, health checks aligned to a common interval, token refreshes on the same schedule, and clients reconnecting after a deployment. All of these produce thundering herds, and all are fixed by adding randomness to the interval.

The practical instruction that follows: any time you write a fixed interval that many instances will share, add jitter proportional to the interval. It costs nothing and removes a class of self-inflicted load spike.