Little's Law in Practice
L = λW — concurrency equals arrival rate times latency — and the reason a slowdown becomes an outage.
Definition
L = λW. The average number of items in a system equals the average arrival rate multiplied by the average time each spends there. It holds for any stable system regardless of distribution, which is what makes it so widely applicable.
The three uses that matter
1. Sizing pools. At 500 requests per second with 200 ms average latency, concurrency is 100. Your thread pool must accommodate 100, plus headroom, or requests queue.
2. Explaining cascading failure. This is the important one. A dependency slows from 50 ms to 3 s — a 60-fold increase in W. At constant λ, L rises 60-fold. The caller's pool, sized for the old concurrency, exhausts immediately. Once exhausted, the caller cannot serve any request, including ones that never touch the slow dependency.
That is the mechanism by which one dependency's slowdown becomes a total outage, and it is why bulkheads — separate pools per dependency — are the highest-value mitigation. It also explains why the initiating fault is usually far smaller than the incident.
3. Reasoning about queues. A queue with 10,000 items and 100 items per second of throughput will take 100 seconds to drain, and any item arriving now waits 100 seconds. This makes queue age the meaningful metric rather than depth.
The related non-linearity
Little's Law is linear, but queueing delay is not. As utilisation approaches 100%, waiting time rises sharply: roughly 4x the service time at 80% utilisation, 9x at 90%, 19x at 95%. This is why:
- Running systems at high utilisation "for efficiency" destroys latency.
- Headroom is a latency feature, not waste.
- A system at 70% utilisation degrades gracefully under a spike and one at 90% falls off a cliff.
Practical applications
- Derive pool sizes from measured λ and W rather than from a default.
- Predict the effect of a latency change on concurrency requirements before it happens.
- Set concurrency limits at what the downstream can absorb.
- Estimate drain time for a backlog during recovery planning.
Failure scenarios
- Pools sized for normal latency, with no headroom for a dependency slowing down — which is the most common incident in distributed systems.
- Targeting high utilisation, producing unpredictable latency.
- Queue depth alerts instead of queue age alerts.
Interview question
"A downstream call goes from 50 ms to 3 seconds at constant traffic. Using Little's Law, explain what happens to the caller and why services that do not use that dependency also fail."