A service under load has high latency and low CPU. The team increases the database connection pool and it gets worse. Why?

Why increasing the pool made it worse

The database has an efficient concurrency level, and exceeding it reduces throughput. More concurrent queries means more contention for locks, buffers, CPU and I/O, more context switching, and more memory per connection. Beyond the efficient point, adding concurrency increases the time each query takes by more than it increases parallelism — so total throughput falls.

The application sees slower queries, concludes the pool is too small, increases it further, and the system degrades further. This is one of the most reliable self-inflicted performance spirals available.

The correct model

A smaller pool with a queue in front of it. Requests wait in the application for a connection rather than overwhelming the database. Total throughput is higher, individual query latency is lower and more predictable, and the queue makes the constraint visible.

The right size is usually far smaller than intuition suggests — often a small multiple of the database's core count rather than one connection per concurrent request. The pool exists to keep the database busy, not to avoid ever waiting.

The diagnostic that identifies it

Separate queue time from service time in the application's own metrics. High latency with low service time means requests are waiting for a connection: that is a queueing problem, and whether the fix is more connections or fewer depends on whether the database is already saturated.

If the database's throughput is flat while its concurrency rises, it is past its efficient point and the pool should shrink. If throughput is still rising with concurrency, the pool can grow.

The related failure at the fleet level

Pool size is per instance and the database sees the sum. Twenty instances with a pool of fifty is a thousand connections, which may be well past the database's limit even though each instance's configuration looks modest. Pool sizing must be reasoned about at the fleet level, and autoscaling the fleet silently scales the connection count — which is how a scaling event becomes a database incident.

The general principle

Bounded concurrency at every shared resource, with queueing in front of it and visibility into the wait. Unbounded concurrency converts a resource limit into a collapse, and it does so precisely when load is highest.