Web & API systems Simulation Intermediate 124 parts 15 Oct 2026

Anatomy of a rate limiter

Four limiters, one stream of requests. Fire a burst, straddle a window boundary, flood them, and watch what each lets through.

"Five requests a second" is not one rule but at least four, and they disagree about bursts. This model puts a fixed window, a sliding window log, a token bucket and a leaky bucket side by side, configured with the same limit and fed the same Poisson stream plus whatever bursts you fire. Every request is a steel ball launched down all four lanes; the limiter's own state moves in front of you, as a peg rack and a clock, a tape of timestamps, a tube of token discs, a trough and a star wheel. The tables read the same simulation: the most each rule has ever let through in a half second and a window, the delay the leaky bucket adds, and how refused clients that retry turn one request into several calls.

The machine

A hopper sends each request, a steel ball, down four parallel lanes at once. At each lane's gate a limiter decides: an admitted ball rolls to a tray, a refused one drops into a bin. The fixed window is a peg rack that fills and a clock that empties it at each boundary; the sliding log is a tape on which one peg per admitted request slides toward expiry; the token bucket is a tube of discs, drained by requests and refilled by a valve; the leaky bucket is a trough that queues balls and a star wheel that releases one every 1/r seconds. A dial per lane reads its admitted rate.

What is real. Four limiters and a discrete-event rig, checked by test.mjs before the page is built: the token bucket decision for decision against the generic cell rate algorithm, the sliding log against a brute-force count, the fixed window against a window-id table and the leaky bucket against a tick-index recurrence, over tens of thousands of random arrivals; the guarantees each rule makes (B + r·Δ for the bucket, L for the sliding window) by brute force over every interval of their output; conservation of requests; a sustained flood settling at r; the 2L boundary burst; and retry amplification in closed form. Every dial, bar and table on the page reads the same rig.

What to take away

  • A token bucket of B tokens refilled at r lets through at most B + r·Δ requests in any span of Δ seconds: a burst allowance and a sustained rate in one rule.
  • A fixed window forgets at its boundary, so a client can send 2L requests in an instant across it while never exceeding L per window; a sliding window never allows more than L in any W seconds.
  • A leaky bucket shapes rather than polices: its output is spaced at least 1/r apart, and the price is up to L/r seconds of waiting before a full queue refuses.
  • Under sustained overload every limiter settles at r; they differ in how the first burst is treated, not in the steady state.
  • Refused callers that retry multiply the calls a limiter sees (a burst of n into a bucket of B, retried k times, becomes n + k(n − B) calls), and jitter flattens the retry peaks.

Sources