A developer platform receives thousands of simultaneous builds after a popular framework release. How should build queues, worker pools, priority scheduling, concurrency limits, caching and backpressure prevent capacity collapse?

Why builds are their own workload class

Builds are bursty, expensive, long-running, untrusted and highly cacheable — five properties that share nothing with serving customer traffic. They belong in a separate pool with their own capacity policy, their own queue and their own failure domain. Interleaving them with the request path means a build spike degrades the platform's API.

The controls

• Per-customer concurrency limits, so one account cannot occupy the whole pool. Primary fairness control.
• Per-customer queues with fair scheduling, not one global FIFO — otherwise a customer pushing fifty commits delays everyone who pushed one.
• Deduplicate identical builds. The same commit built twice concurrently is common with retries and webhook duplicates, and collapsing them is free capacity.
• Cancel superseded builds. If three commits land on a branch while the first is queued, only the last one usually matters. This is often the single largest capacity saving during a burst and it costs almost nothing to implement.
• Aggressive layer and dependency caching, since a large fraction of build time is fetching and compiling the same dependencies. A cache hit converts a five-minute build into a thirty-second one, which is a capacity multiplier rather than a latency improvement.
• Admission control with honest feedback. When the queue implies an unacceptable wait, tell the user their position rather than accepting silently. A build queued invisibly for twenty minutes is worse than a rejection.

The isolation requirement

Build workloads run untrusted customer code, which means they need genuine isolation — separate sandboxes or microVMs, no shared credentials, network egress controls, and a filesystem that does not persist between builds except through an explicit cache. That isolation costs startup time, which is in tension with the latency requirement, and the usual resolution is a pool of pre-warmed sandboxes claimed on demand.

What backpressure looks like here

Not dropping builds — a dropped build is a broken developer workflow. It is admitting them with an honest estimate and prioritising: interactive builds triggered by a push ahead of scheduled or batch rebuilds, paying customers ahead of free tier, and short builds ahead of long ones where fairness allows, since short-job-first minimises average wait.