pattern

Token Bucket

A rate-limiting algorithm holding a replenishing allowance of tokens, permitting controlled bursts while bounding the sustained rate.

rate-limitingalgorithmsbursts

Tokens refill at a fixed rate up to a maximum capacity; each request consumes one. A client that has been idle accumulates tokens and may burst up to the bucket size; a client at sustained load is limited to the refill rate.

That burst tolerance is why it is usually the right choice: real traffic is bursty, and an algorithm that rejects a short burst a client could legitimately afford produces unnecessary failures.

Compared with the alternatives: fixed window is trivial and allows a double burst across the window boundary; sliding window fixes that with more state; leaky bucket smooths output to a constant rate, which suits protecting a downstream that cannot absorb bursts at all.

The implementation requirement in a multi-instance deployment is a shared counter — otherwise the effective limit is the configured limit times the instance count, which is the most common rate-limiting bug.