A checkout handler takes a pooled database connection at the start of the request and releases it at the end. In between it calls a partner API that takes 300 ms at p50. Traffic is 400 requests per second and the SQL itself takes 15 ms. Roughly how large must the pool be and what is the better answer?

The assumptions, stated

Little's Law (Little, 1961) sizes a pool: L = λW. The trap is in W. W is the time a connection is unavailable to anyone else, which is the hold time, not the query time. Assume the handler checks out at entry, performs 15 ms of SQL in total, waits 300 ms for the partner, and returns the connection at exit.

The arithmetic

• Hold time ≈ 15 ms of SQL + 300 ms of partner call + roughly 5 ms of handler overhead ≈ 320 ms.
• L = 400 × 0.320 = 128 connections held at steady state, before any safety factor.
• Release the connection before the partner call and reacquire after: W = 15 ms, so L = 400 × 0.015 = 6 connections, and 10–12 with headroom.

The same traffic, the same queries, and a 20× difference in the pool the database must support.

The failure mode is distinctive. When the partner slows down, every request holds its connection for longer, the pool empties, and requests begin to fail on checkout timeouts while the database reports low CPU and sub-millisecond query times. The outage looks like a database problem and is not one.

Which assumption dominates the error

Not the SQL. Doubling the 15 ms moves the first answer from 128 to 134, about 5%. The partner's tail does all the damage: at a p99 of 2 s the hold time for that 1% is 2 s, and sizing for a period where the partner's whole distribution shifts to 2 s needs 400 × 2 = 800 connections. You are sizing a pool against the latency distribution of a system you do not control and cannot see.

Then the second-order limit bites. A PostgreSQL backend process costs on the order of 5–10 MB of private memory, so 128 connections on each of 40 instances is roughly 5,000 backends and tens of gigabytes before any query runs. A transaction-mode pooler such as PgBouncer is the standard fix, and it only works if the application stops holding a connection across the partner call, because transaction-mode multiplexing returns the connection between transactions.

What the number rules in or out

Choose this rule and keep it: never hold a pooled resource across a call whose latency you do not control, unless that call is local and bounded. It rules out the convenient "open a transaction for the whole handler" pattern, and it rules in an explicit sequence: read what you need, release, call the partner, reacquire, write the result.

If the write must be atomic with the partner's outcome, the answer is not a longer-held transaction — it is a state machine: persist an intent, call the partner, record the outcome idempotently, reconcile anything stranded. That costs a table, a retry path and a reconciliation job.

When this is the wrong answer

If the external call is 5 ms inside your own data centre, hold the connection. The hold-time difference is 15 ms versus 20 ms, the pool moves from 6 to 8, and splitting the handler into two transactions buys nothing while giving up the simplicity of one atomic unit. The pattern earns itself when the external latency is an order of magnitude above the local work, or when its tail is unbounded.

Common weak answers

• "Set the pool to 200 to be safe." That is the 128 answer with extra memory, and it pushes the queue into the database where it is harder to see.
• "Add a pooler." Useful, and it cannot multiplex a connection that one request holds for 320 ms.
• "The database is idle, so it is not the bottleneck." The bottleneck is the pool, and an idle database with saturated checkout queues is its exact signature.