Utilisation and Queueing Delay
The non-linear relationship by which waiting time grows as utilisation approaches one, explaining why systems degrade suddenly rather than gradually.
For a simple queue, waiting time scales with 1/(1 − utilisation).
The numbers are the point. At 50% utilisation, expected wait is about one service time. At 80%, four. At 90%, nine. At 95%, nineteen. At 99%, ninety-nine.
This is why systems seem fine and then collapse. Between 50% and 70% almost nothing happens; between 90% and 95% latency doubles. Capacity plans built on a linear mental model consistently under-provision, because the last increment of utilisation is where all the latency lives.
Three practical consequences:
Target utilisation well below saturation — commonly 60–70% for latency-sensitive services. The apparently wasted headroom is buying the flat part of the curve.
Variability makes it worse. The formula above assumes a specific arrival and service distribution. Bursty arrivals and highly variable service times produce longer queues at the same utilisation, which is why a service with a 500× spread in request cost needs more headroom than its average suggests.
More servers beat faster servers for queueing, because pooled capacity absorbs variance better — the reason a single shared queue outperforms per-server queues.