Throughput has plateaued and adding instances does not help. What are the candidate bottlenecks and how do you identify which?

What is being tested

Systematic bottleneck identification, and the recognition that adding capacity to a non-bottleneck does nothing but increase the bill.

The reasoning

Throughput is bounded by the single narrowest resource. Adding instances increases capacity in one dimension; if the constraint is elsewhere, nothing changes. The fact that instances do not help is itself diagnostic: the bottleneck is shared.

The candidates, in rough order of likelihood

1. The database. The most common shared bottleneck: connection limit reached, CPU saturated, lock contention, or disk I/O. More application instances means more connections against the same limit, which frequently makes it worse.

2. A shared downstream service or third-party API, including its rate limit. Your limit is their limit.

3. A connection pool at some layer — the application's pool to the database, a pooler's pool, a proxy's connection limit.

4. A lock. A distributed lock, a row-level hot spot, or a global lock in a shared library. All instances serialise on it, so more instances just means more waiting.

5. Network — bandwidth on a link, packets per second, or a NAT gateway's connection limit.

6. A single-threaded component in the path: a coordinator, a sequencer, a leader.

7. The load generator, if this is a test. Measuring your own test harness is embarrassing and common.

How to identify which

Measure saturation, not utilisation. Utilisation says how busy; saturation says how much is waiting. A CPU at 100% with an empty run queue is fully used and fine; a disk at 40% with a deep I/O queue is the bottleneck. Look at queue depths and wait times.

Follow a request with a trace. Where is wall-clock time spent? This points at the waiting, which is faster than inferring it from resource metrics.

Check the logical resources. Physical resources are monitored; connection pools, thread pools, lock waits and third-party quotas usually are not. When CPU, memory, disk and network all look fine and the system will not go faster, it is one of these.

Verify by relieving it. If you believe it is the database, cache the hottest query and see whether throughput moves. If it does not, you were wrong — which is common and worth discovering in an hour rather than a quarter.

The property that makes this iterative

Relieving a bottleneck moves it. Fix the database and the application tier saturates; fix that and the network does. Each round must begin with re-identifying the constraint, and a plan that optimises three things simultaneously usually improves one.

What a strong answer adds

That the bottleneck is rarely where the symptom is. A slow API endpoint is frequently blocked on a connection pool exhausted by an unrelated slow dependency — so the endpoint everyone is looking at is a victim, not a cause.