Connection Fan-In

A traditional application server holds a small pool of long-lived connections and multiplexes many requests across them. Ephemeral compute inverts this: many short-lived instances, each wanting its own connection, with no coordination between them.

A thousand concurrent invocations attempt a thousand connections against a database whose practical ceiling is far lower, and where each connection costs memory whether or not it is executing anything.

Why it matters

The failure is abrupt rather than gradual. Connections are fine, then exhausted, then the database rejects everything — including the traffic it could easily have served. And it is triggered by success: a spike in usage produces a spike in instances, which produces the exhaustion.

It is also counter-intuitive, because the compute tier scaled correctly. Serverless solves compute elasticity and creates a fan-in problem at every stateful dependency.

Implementation patterns

• An external connection pooler between ephemeral clients and the database, multiplexing many client connections onto few server connections. Its absence is the actual cause.
• Transaction-level rather than session-level pooling, which allows far higher multiplexing at the cost of session state — prepared statements, temporary tables, session variables, advisory locks. That constraint must be communicated, because it breaks code that previously worked.
• A hard connection cap per tenant, so one customer's runaway invocation count cannot exhaust a shared pooler.
• Short statement timeouts, since a slow query now holds a pooled connection many clients are waiting for.
• The same intermediary pattern at every stateful dependency: a queue in front of a rate-limited third party, a shared gateway holding upstream connections, a cache client with a bounded pool.

Industry example

Platforms such as Supabase, which pair a managed Postgres with customers running serverless functions, meet this immediately and at scale — the product's two halves have incompatible connection models unless an intermediary sits between them. The pooler is not an optimisation in that architecture; it is a required component.

The same shape appears with any ephemeral compute against any per-connection-cost dependency: edge functions calling a database, autoscaled containers against a rate-limited API, batch workers against a legacy system with a session limit.

Failure scenarios

• No pooler, producing abrupt exhaustion at the first traffic spike.
• Session-level pooling only, so multiplexing is limited and the problem returns at higher scale.
• Transaction pooling adopted without communicating the constraints, breaking prepared statements and advisory locks in ways that appear as intermittent application bugs.
• No per-tenant cap, so one customer exhausts a shared pooler.
• The pooler becoming the single point of failure, without its own redundancy.

Trade-offs

A pooler adds a hop, a component to operate, and a new failure domain, and transaction-level pooling removes capabilities application developers expect to have.

The alternative for some workloads is to avoid the connection model entirely with a stateless data API over HTTP, where the request model matches the compute model. That sidesteps the problem rather than solving it, at the cost of different consistency guarantees, a less capable query interface, and a per-request overhead that is fine for many workloads and wrong for transactional ones.

Interview question

"Your serverless functions work perfectly in testing and exhaust the database connections in production at peak. Explain the mechanism, give me the fix, and tell me what will break in the application code when you apply it."