Bottleneck Analysis
Finding the single resource that limits the system, because improving anything else changes nothing.
Definition
Every system has exactly one binding constraint at any moment. Throughput is determined by it, and optimising any other component produces no improvement — a fact that is obvious when stated and routinely ignored in practice.
The method
1. Measure saturation, not utilisation. Utilisation says how busy something is; saturation says how much work is waiting. A CPU at 100% utilisation with no run queue is fully used and not a bottleneck; a disk at 40% utilisation with a deep I/O queue is. Queue depth and wait time are the signals.
2. Check every resource class systematically. CPU, memory (including allocation rate and garbage collection), disk I/O, network, and — most often the answer in a distributed system — a logical resource: a connection pool, a thread pool, a lock, a semaphore, a per-account API quota.
3. Follow the request. A trace showing where wall-clock time is spent finds the bottleneck faster than resource metrics, because it points at the waiting rather than at the busyness.
4. Verify by relieving it. If you believe the database is the bottleneck, add a cache for the hottest query and see whether throughput moves. If it does not, you were wrong — which is common and worth discovering in an hour rather than a quarter.
The property that makes this iterative
Relieving a bottleneck moves it. Fix the database and the application tier saturates. Fix that and the network does. Each round of work must begin with re-identifying the constraint, and a plan that optimises three things at once usually improves one.
There is a corollary worth internalising: the bottleneck is rarely where the symptom is. A slow API endpoint is frequently blocked on a connection pool that is exhausted because of an unrelated slow dependency.
The logical bottlenecks people miss
Physical resources are monitored; logical ones usually are not. Connection pools, thread pools, lock contention, per-account cloud service limits, a single-threaded coordinator, a global lock in a shared library, and a downstream provider's rate limit. When CPU, memory, disk and network all look fine and the system will not go faster, it is one of these.
Failure scenarios
- Adding capacity to a non-bottleneck, producing a bigger bill and identical performance.
- Utilisation mistaken for saturation.
- Optimising the average path when the bottleneck is in the tail.
- A bottleneck outside your system — a third-party rate limit — treated as an internal problem.
Interview question
"CPU, memory, disk and network all look healthy, and the service will not go faster. What are you looking for now?"