A serverless API works in testing and fails under load with connection errors. The database is at 5% CPU. Explain and fix.

The mechanism

Serverless functions scale by creating independent execution environments, each with its own process and its own connection pool. Two hundred concurrent invocations means up to two hundred separate connections, and if each opens a modest pool it is far more.

A managed database typically allows connections in the low hundreds — the limit scales with instance size, not with your concurrency. It refuses new connections while sitting at 5% CPU, because connections are a memory and process constraint rather than a compute one. That mismatch is exactly why the CPU metric is misleading here.

In testing, concurrency never reached the limit, so it worked.

The fix

A connection proxy — RDS Proxy, Cloud SQL Auth Proxy, PgBouncer. Clients connect freely; the proxy maintains a small pool of real database connections and hands them out per transaction.

Two things to know before adopting one:

Pooling mode changes semantics. Transaction-level pooling — the efficient mode — breaks session-scoped features: prepared statements, temporary tables, advisory locks, SET variables. Consecutive statements may run on different backend connections, so some applications need changes.

It is a component that must be highly available and adds a small latency hop. Managed proxies partly repay this by holding client connections open during a database failover, which shortens failover impact.

The other things worth doing

Reduce pool size per function instance to one or two. A function handles one request at a time; a pool of ten per instance is pure multiplication.

Reserve concurrency on the function, capping how many instances can exist — which bounds the demand rather than only absorbing it.

Consider whether serverless is right here. Sustained request-driven load against a database is the workload serverless is worst at, and a container service with a proper pool has none of this problem.

Why the other options are wrong

Scaling up raises the limit and does not remove the mismatch — it postpones it. A security group would fail in testing too. A missing index would show as latency and high CPU, not connection refusals.