Every relational database serves far more clients than it can hold sessions for, so something between the application and the database must ration connections. This guide reconstructs how that ration line is drawn in production at GitLab, Notion, Figma, OpenAI, Cloudflare and others, what it costs, and the four ways it fails.
The problem, who has solved it in production, and the finding that reorganises how you should think about it.
State the problem without naming a product. A relational database executes work on a small, fixed number of operating system processes, and each client that might issue a query wants a standing, stateful session held open on its behalf. Demand for sessions grows with the size of the application fleet, and with serverless it grows with request volume itself, while the supply of sessions the engine can hold barely grows at all. The largest hosted Postgres plans topped out around 500 connections when Brandur Leach surveyed them in 2018, and the picture inside the engine explains why: Andres Freund's 2020 measurements found that a single active connection slows by more than 2x in the presence of many idle ones, because every transaction rebuilds a snapshot that, before Postgres 14, touched shared state for every open connection. Meanwhile Supabase demonstrated one million connected clients served by 400 actual database connections. The gap between those two numbers, three to four orders of magnitude, is the territory of this guide.
The teams that have written about crossing this gap in production include GitLab, Notion, Figma, OpenAI, Cloudflare, ClickHouse and Supabase; the published failures come from Red Hat's Quay.io, GitHub, Honeycomb and GitLab again. Every one of them converges on the same two-line architecture drawn in Figure 1, and every one of them has hit at least one of the four failure classes in section 4.
The finding that surprised me, and that reorganises the topic: the hard part stopped being connection reuse years ago. Reuse is a solved mechanism you can install in an afternoon. What the most mature deployments are actually engineering at the pool is admission control, the decision of who waits, in what order, and who is refused when the budget is short. Figma's 2026 account is the clearest statement: they replaced the standard pooler with PGKeeper, a service whose headline features are queue scheduling (CoDel plus adaptive LIFO), token-bucket limits on connection churn, and weighted fair sharing across traffic classes, and they report it prevented more than 20 would-be incidents in a single quarter. The pool, it turns out, is where load management lives, because it is the one chokepoint every query already passes through. The theory agrees: a bounded pool converts an open arrival process into a partly closed one, which is precisely the regime that Schroeder, Wierman and Harchol-Balter showed behaves qualitatively differently under overload.
Scope. This guide covers the connection path between an application fleet and a single-writer relational database: pool sizing, pooler topology, pooling modes and their semantics, and the published failure record, with Postgres as the best-documented case and MySQL appearing through GitHub's ProxySQL and Quay.io's incidents. It deliberately does not cover read-replica routing policy, sharding (a separate guide in this series covers resharding), driver-internal retry semantics, or serverless database engines that dissolve the connection limit inside the service, such as the multiplexing described in the Aurora Serverless paper, which appears here only as a boundary marker.
The common shape across GitLab, Notion, OpenAI, Figma and Supabase, with the points where they diverge and why.
The reference architecture has three tiers of scarcity, and each published system draws
them the same way. At the top, every application process keeps a private pool measured in
single digits to low tens; the HikariCP project's long-standing guidance is
"you want a
small pool, saturated with threads waiting for connections", sized near
cores × 2 + spindles for the database host, and its most-cited exhibit is
an Oracle demonstration in which shrinking the pool, with no other change, cut application
response times from roughly 100 ms to roughly 2 ms. In the middle sits a shared pooler tier
speaking the database's wire protocol, run in transaction mode so a server connection is
lent out only for the duration of one transaction. At the bottom, the database itself is
given a deliberately small ceiling: Notion
capped each Postgres instance at
200 connections during its 2023 re-shard, and OpenAI
fronts every one of its roughly 50
read replicas with PgBouncer while pushing more than a million queries per second
through the fleet.
Why transaction mode wins the middle tier is stated plainly in GitLab's runbook: their clients "use long-lived connections to execute transactions from different requests spread over time", so a session-mode pooler would hold a server connection hostage to every idle client. Transaction mode turns idle clients from a cost on the database into a cost on the pooler, and poolers are far better at holding idle sockets: Supavisor's repository documents 250,000 idle client connections on a single 16-core node, where the same quarter-million connections held open on Postgres would consume, at Freund's measured 1.3 to 7.6 MiB each, somewhere between 300 GiB and 1.8 TiB of memory and would degrade every active query through snapshot construction besides. That arithmetic is mine; the per-connection figures are Freund's 2020 measurements.
The divergence points are three. First, placement: GitLab runs
three
dedicated pooler hosts in front of the primary but co-locates three pooler processes on each
replica host, a split that follows write traffic's need for independent failover.
Cloudflare moved the pool off the customer's premises entirely; Hyperdrive holds pools near
the database and lets edge clients skip
the seven
round trips a fresh connection costs (one TCP, three TLS, three auth). Second,
process model: PgBouncer is single-threaded by design, so GitLab, ClickHouse and Crunchy
Data all scale it by running several processes behind SO_REUSEPORT;
ClickHouse
measured one process peaking at about 87k transactions per second and a per-core fleet
reaching about 336k. Supabase and Figma instead replaced the pooler with a
multi-threaded one. Third, what the pooler is for: at Figma the replacement,
PGKeeper,
is explicitly a load-management service, with pool warming, token-bucket limits on
connection creation and teardown, and two-stage admission control; connection reuse is
almost incidental.
One mechanism deserves its own paragraph because it explains a decade of production errors: protocol-level prepared statements. A prepared statement lives in one server session. Under transaction pooling, consecutive transactions from the same client land on different server connections, so the statement a client prepared is marooned on a connection it may never see again. For years the only fix was to disable server-side preparation entirely and forfeit its performance; PgBouncer 1.21 (October 2023) finally tracks prepared statements per client and re-prepares them on whatever server connection the transaction lands on, and Cloudflare's Hyperdrive implements the same per-client, per-origin-connection bookkeeping. The release notes put the recovered throughput at 15 to 250 percent depending on workload.
Holds a handful of connections per application process and queues threads that want one. Its size is set by the database host's parallelism, not by client count.
Doctrine: HikariCP wiki. In production at effectively every JVM, Rails and Go shop.
Lends a server connection for one transaction at a time, so thousands of idle clients cost the database nothing. The price is session semantics; see section 3.
A deliberate cap in the low hundreds per instance, because idle server connections tax memory and, pre-Postgres 14, every snapshot. The budget is the scarce resource the whole stack rations.
Measured cost: Freund, 2020; cap: Notion, 2023.
Queue scheduling, rate limits on connection churn, and fair sharing across traffic classes, placed at the pool because every query already passes through it.
The built case: Figma's PGKeeper, 2026.
Because every client goes through it, the pooler doubles as the switch for zero-downtime failover: pause traffic, drain in-flight queries, repoint, resume.
Used for a 32-to-96-instance migration at Notion, 2023.
Client connections waiting, server connections active, and checkout wait time are the three signals; GitLab alerts on pooler client-connection saturation as a first-class incident trigger.
Incident opened from that signal: GitLab, 2022.
Each fork as the published record shows it: what was chosen, what was rejected, the stated reason, and the condition that flips the answer.
max_prepared_statements; the same design in HyperdriveprepareThreshold=0 and equivalents), documented in driver issues back to 2017One decision deserves narrative rather than a table: whether to solve the problem inside
the database. Raising max_connections is the intuitive move and every measured
source argues against it. Freund's analysis concluded the binding constraint was
snapshot
scalability, not memory, and his own
Postgres 14 commits
attacked exactly that, calling the old snapshot construction "the most significant source of
GetSnapshotData() scaling poorly on larger systems". Postgres 14 raised the ceiling; it did
not remove it. The engine still spends real memory and scheduling capacity per connection,
which is why the AWS idle-connection measurements below remain relevant six years on, and
why every large deployment in this corpus still rations. Nobody in the published record
scaled out of this problem with a configuration change.
| Decision | Chosen | Rejected | Because | Evidence |
|---|---|---|---|---|
| Pooler tier | Insert when connections hit thousands | Day-one pooler | Cost precedes benefit at small scale | Figma, 2023 |
| Pool mode | Transaction | Session | Idle clients must not pin servers | GitLab runbook |
| Pooler scaling | SO_REUSEPORT process fleet | Multi-threading the pooler | Threading remains an open issue since 2024 | pgbouncer #1021 |
| Prepared statements | Pooler-side tracking (1.21+) | Disable preparation | 15–250% throughput at stake | Release 1.21.0 |
| Raise max_connections | Ration and queue instead | Big connection caps | Idle connections tax every active query | Freund, 2020 |
| Load management | Admission control at the pool | Per-service client throttles alone | The pool sees all traffic and the true budget | Figma, 2026 |
Five published incidents and one chronic failure, grouped into the four classes the record actually contains.
Group the incidents and four classes emerge. Storms: unbounded clients open connections faster than the database can absorb them. Recovery herds: the outage ends, and simultaneous reconnection and cold caches re-create it. Semantic leaks: transaction pooling silently changes session semantics and the application finds out in production. Pooler saturation: the ration line itself becomes the bottleneck. The first two are capacity failures with an open-loop signature; Marc Brooker's summary of the queueing theory applies to both: congestive collapse is the characteristic risk of open-loop systems, and retries add arrival traffic exactly when completions slow.
prepared statement "S_1" does not exist even with preparation nominally disabled, from 2017 onward; session-level SET statement_timeout leaks to other clients' transactions. JP Camara's 2023 survey concludes "the road to downtime is paved with session level statements".Everything quantitative in the corpus, with where it was measured and when. Treat the ratios as durable and the absolutes as dated.
| Metric | Value | At | Context | As of | Source |
|---|---|---|---|---|---|
| Idle-connection tax on one active query | >2× slower | Postgres 13 | 10,000 idle connections, measured pre-snapshot-fix | 2020 | Freund |
| True memory per idle connection | 1.3–7.6 MiB | Postgres | Proportional set size; 1.3 with huge_pages, not the folkloric tens of MB | 2020 | Freund |
| Throughput loss from 1,000 idle connections | −8.7% / −46% | AWS RDS | db.m5.large (2 vCPU, 8 GB); mixed load / select-only | 2020 | AWS |
| Response time from shrinking the pool alone | ~100 ms → ~2 ms | Oracle demo | No other change; the small-pool doctrine's exhibit | n.d. | HikariCP wiki |
| One pooler process, connection ceiling | ~10,000 / ~1,000 active | Crunchy Data | Planning guidance for single-threaded PgBouncer | 2022 | Crunchy |
| One pooler process, throughput ceiling | ~87k tps | ClickHouse | Degrades to 77k at 256 clients; single core saturated | 2026 | ClickHouse |
| SO_REUSEPORT fleet, same host | ~336k tps (~4×) | ClickHouse | Multiple PgBouncer processes, one per core | 2026 | ClickHouse |
| Idle clients per purpose-built pooler node | 250,000 | Supabase | 16 cores, 64 GB; Elixir, multi-threaded | 2023 | Supavisor README |
| Multiplexing ratio at benchmark extreme | 1,000,000 : 400 | Supabase | 20k QPS through 400 server connections, 2 pooler nodes | 2023 | Supabase |
| Hosted Postgres connection caps | 500 / 20–25 | Heroku et al. | Largest plans / smallest plans, at survey time | 2018 | Leach |
| Per-instance server connection budget | 200 | Notion | Deliberate cap during 96-instance re-shard; 8 conns per pooler per shard | 2023 | Notion |
| Largest published single-primary deployment | ~50 replicas, >1M QPS | OpenAI | PgBouncer per replica; connection time 50 ms → 5 ms | 2026 | OpenAI |
| Round trips to a fresh connection | 7 | Cloudflare | 1 TCP + 3 TLS + 3 auth, before the first query | 2025 | Cloudflare |
| Prepared statements, throughput recovered | +15–250% | PgBouncer | Workload-dependent; feature landed October 2023 | 2023 | Release notes |
| Incidents prevented by pool-level admission control | >20 in one quarter | Figma | Company-reported count for PGKeeper, Q4 2025 | 2026 | Figma |
Measured: the Freund, AWS, ClickHouse and Supabase rows come from described benchmark setups. Claimed: Figma's incidents-prevented count and OpenAI's QPS are company-reported without a public method; treat them as claims from credible primaries. Derived: the 300 GiB to 1.8 TiB figure in section 2 is my arithmetic (250,000 × 1.3 to 7.6 MiB) and is labelled as such where it appears. Unknown: nobody in this corpus publishes the money cost of a pooler tier (instances, cross-AZ traffic, licensing for managed proxies), so any cost model you build will rest on your own cloud bill, not on public evidence. The dominant variables, from the architecture: pooler cores (one per ~87k tps), pooler memory (idle clients), and one extra network hop on every query.
Every source behind this page, graded. The ledger shipped beside this file (sources.md) carries the exact supporting quotes and the retrieval method for each.
A fleet of workers on default connection settings produced tens of thousands of connections and locked MySQL; load tests later located the real ceiling near 10,000.
Three incidents in a month traced to the pooling tier; the third began when active connections "crossed a critical threshold that changed the behavior" of ProxySQL.
A 68-minute total outage whose recovery had to be engineered: ingest was refused with 5xx to protect the database connection budget while caches warmed.
A modest database slowdown multiplied concurrent cache refreshes, each holding a connection; crash loops cleared caches and re-hit the database at 90% CPU.
The pooler's own client-connection budget saturated on the primary path. The public review's fields were left incomplete; the incident record and its saturation metric are the evidence.
The design problem stated by the operator: latency degrades when the web fleet grows or max_client_conn rises, because the single-threaded pooler pegs one core.
The maintainer's own case for threading, recording exactly what the SO_REUSEPORT workaround costs: un-shared pool limits, per-process statistics, awkward setup. Open as of September 2026.
The small-pool doctrine and its formula, with the Oracle demonstration of a 50x response-time improvement from shrinking the pool alone.
The commit message names the mechanism of the connection tax: snapshot construction caused "many cache misses" and was "the most significant source" of poor scaling on large systems. Postgres 14 deferred that work.
The merged prepared-statement tracking, called "probably one of the most requested features" in the project's history, worth 15 to 250 percent throughput.
The parallel prepared-statement implementation, closed after a year when review surfaced hash-collision and protocol defects; the recorded argument about why this feature is hard.
With prepareThreshold=0, autoCommit=false and a fetchSize, the JDBC driver still created a named statement, and transaction pooling produced "S_1 does not exist".
The deployed topology: three dedicated pooler hosts before the primary, three co-located processes per replica, transaction mode justified by long-lived idle client connections.
A clustered, multi-threaded pooler built because the standard one "is single-threaded, making it difficult to scale"; documents 250k idle connections per 16-core node and zero-downtime scaling as a goal.
One primary, roughly 50 replicas, a pooler in front of each, more than a million queries per second; connection time fell from 50 ms to 5 ms behind the pooler.
A described benchmark: one million client connections across two pooler nodes, triaged into 400 database connections at 20k QPS.
The measurements behind this whole topic: the 2x idle tax, the memory myth (1.3 to 7.6 MiB, not tens), and the identification of snapshots as the binding constraint.
The canonical first move, dated: pooler inserted when app connections reached the thousands, alongside replicas and vertical partitioning, under 3x annual traffic growth.
The endgame account: outgrew the standard pooler on threading, load management and extensibility; built a Go service with CoDel plus adaptive-LIFO admission control and connection-churn rate limits; reports more than 20 incidents prevented in Q4 2025.
Explicit budgets everywhere: 96 instances at 200 connections each, four pooler clusters of 24 databases, eight connections per pooler per shard, and pause/resume as the zero-downtime failover mechanism.
The connection-setup arithmetic (seven round trips) and a transaction-mode pooler at the edge with per-client, per-origin prepared-statement tracking.
Measured single-process ceiling (~87k tps, degrading past saturation) and the SO_REUSEPORT fleet's ~4x recovery, with the one-core-of-sixteen framing.
The operator's planning numbers: roughly 10,000 connections per process, roughly 1,000 concurrently active, one core forever.
The hosted ceilings (500 at the top, 20 to 25 at the bottom) and the practitioner techniques: minimum viable checkout time, releasing connections around slow external calls.
Worked examples of transaction-mode semantic leaks: statement_timeout crossing clients, advisory locks, driver quirks. "The road to downtime is paved with session level statements."
The queueing frame applied by a practitioner: closed loops queue and stay stable, open loops collapse under overload, and retries add arrivals precisely when service slows.
The formal result underneath this guide: open and closed workload models diverge vastly under load, most designers ignore the distinction, and eight principles govern the difference.
The boundary marker: inside a managed serverless engine, connection and buffer capacity is oversubscribed and reactively managed against a measured definition of "heat", dissolving the fixed budget this guide is about.
By the engineer who landed the PgBouncer implementation: how PgBouncer, Odyssey, pgcat and Supavisor differ in prepared-statement design and performance. Claim taken from the published abstract; this build environment could not retrieve timestamps.
The conference companion to the OpenAI case study: pooler management, replica scaling and long-query handling at millions of QPS. Claim taken from the published session description.
The semantics in the operator's own words, including the warning that transaction mode forbids session-based features, and the defaults (pool size 20, client cap 100) that everyone tunes upward.
Vendor-run but methodical: 1,000 idle connections on a db.m5.large cut mixed throughput 8.7% and select-only throughput 46%, by eating the OS page cache.
The managed proxy's answer to session state: detect it and pin the client to a dedicated connection until the session ends, silently trading multiplexing for correctness.
Six rungs from an evening experiment to an operable ration line. The crossing from toy to real happens at rung four.
Local Postgres, default configuration. Write a script that opens connections until the server refuses, watching memory per backend as you go.
Done when: you have seen the refusal error text and can state your measured per-connection memory. Teaches: the budget is real and smaller than folklore says.
Run pgbench at fixed concurrency, then repeat with 1,000 idle connections held open by a second process. Try it on Postgres 13 and 16 if you can.
Done when: you have two throughput numbers and a delta. Teaches: idle is not free, and the tax changed size in Postgres 14.
One web app, one load generator, pool of 100 versus pool of 10 against the same 4-core database, measuring p99 at saturation.
Done when: the small pool wins at saturation and you can explain why in one sentence. Teaches: the queue belongs in front of the database, not inside it.
Put PgBouncer in the path. Reproduce a session-state leak: SET a timeout on one client and observe another inherit it; prepare a statement and watch it maroon on an old pooler version, then fix it with max_prepared_statements on 1.21+.
Done when: you have produced and then eliminated the marooned-statement error. Teaches: pool mode is an application semantics change.
Under steady load, restart the pooler and the app fleet together. Watch the reconnection herd hit the database, then add jittered backoff and a connection-creation rate limit and run it again.
Done when: recovery no longer re-triggers the overload. Teaches: the open-versus-closed distinction, physically.
Export waiting clients, active server connections and checkout wait time. Set the saturation alert below the threshold you found in rung five's failures, and write the one-page runbook for the alert.
Done when: a synthetic overload pages you before the first user-visible error. Teaches: the pooler is a capacity-planned service with its own SLO.
The queries that actually found this material, grouped by what they surface. The vocabulary is the value: saturation, pinning, session state, SO_REUSEPORT.
pgbouncer postmortem OR "incident review" saturation site:gitlab.com"connection pool" exhausted postmortem outage "we""active database connections" threshold "post-incident""database connection saturation" incident reviewrepo:pgbouncer/pgbouncer is:pr is:closed is:unmerged preparedpgbouncer "multi-threading" issue so_reuseport"prepared statement" "S_1" "does not exist" pgbouncerpostgres commit "snapshot scalability" GetSnapshotData"we replaced pgbouncer" OR "outgrowing pgbouncer""single-threaded" pgbouncer "one core" scale "we""max_client_conn" latency degraded fleet"connections per" instance capped re-shard pgbouncer"idle connections" postgres measured TPS memory impact"open versus closed" workload connection pool collapseRDS Proxy pinning "session state" multiplexingconnection pooler "prepared statements" comparison talkChecked 2026-09-06. This guide was assembled in a sandbox that can fetch only github.com and gitlab.com directly; sources on other hosts were verified through live search retrieval of their exact wording, as recorded per-claim in the accompanying sources.md ledger.