Integration Throughput Ceiling

A merge queue that tests each change against the trunk plus the queue ahead of it, then merges serially, has an arithmetic limit. With a 20-minute run: 3 merges an hour, 24 in an 8-hour working day, 72 around the clock. Sixty engineers merging twice a day want 120, so the queue is short by a factor of five against the hours people actually work.

The number matters because of what happens past it. A queue driven beyond its service rate does not merely get slower; it changes behaviour. Waits reach hours by mid-afternoon, engineers batch changes to avoid queueing, changes get larger, failures get more expensive to diagnose, and the team stops practising continuous integration while still operating a merge queue.

The ceiling also gives an honest team-size threshold: a serial queue with a 20-minute pipeline supports roughly 10 to 12 engineers.

Why it matters

It is the one capacity limit in delivery that is usually invisible until it has already changed how people work. Pipeline duration is watched; merges per day against capacity is not. By the time anyone measures it, the symptom has been absorbed as culture: people hold changes until tomorrow, or open a branch and integrate weekly, and the cause is recorded as a preference rather than a queue.

It is also the limit that pipeline tuning cannot fix. Halving run duration doubles the ceiling, and doubling is not enough at organisational scale, so the ceiling forces a structural choice — batching, test selection, or moving checks out of the gate.

Implementation patterns

• Optimistic batching. Test the top N of the queue as one candidate and merge all N on success. N = 5 gives about 15 merges an hour. On failure, bisect: 3 extra runs for a batch of 5, paid only when something is broken.
• Know the flip condition. Batching wins while the failure rate is low; at roughly a 30% failure rate the bisects dominate and serial is better. So the prerequisite for batching is a trunk whose failures are real, which means flaky tests are quarantined first.
• Change-aware test selection computed from the build graph, running only what a change could affect. On a large repository this routinely takes a 20-minute run to 4 minutes, and it raises the ceiling more per hour of work than anything else.
• Shard across runners. Throughput is partly a purchase; a suite that shards cleanly converts machines into wall-clock time.
• Move slow broad checks out of the gate. Run them post-merge on a schedule with a revert rule. Not every check has to be a gate; some can be a detector.
• Measure the queue, not the pipeline: merges per day against capacity, wait time at p95 by hour of day, and batch failure rate.

Industry example

Every organisation operating a large shared repository has hit this and published some version of the same answer, and the monorepo engineering write-ups from about 2015 onwards describe the same two mechanisms: merge queues with batching and bisection on failure, and build systems whose dependency graph supports running only affected targets, which is the design purpose of graph-based build tools rather than a convenience. The consistent finding from those accounts is that the effective fix was the test-selection graph rather than faster machines, because machines scale the cost linearly while selection removes work entirely.

Failure scenarios

• Queue length growing through the day, with afternoon waits in hours and an empty queue by morning.
• Batching without bisection, so one bad change rejects four good ones and trust in the queue collapses.
• Batching on a flaky trunk, where most batches fail and throughput falls below serial.
• A ceiling raised by removing gates that mattered, trading queue time for production incidents.
• Engineers routing around the queue with direct pushes or long branches, which is the real cost and is invisible in the queue's own metrics.
• Test selection that under-approximates the graph, so an affected test is not run and the trunk breaks anyway.

Trade-offs

When not to use it

Below about 15 engineers a serial queue is simpler, easier to reason about and fast enough, and batching adds a bisection mechanism plus a failure mode where a good change is rejected. Do not build it against a theoretical wait: the signal to act on is engineers saying they are holding a change until tomorrow. The metric is also the wrong lens where the constraint is elsewhere — if the real delay is a two-day review queue, raising integration throughput changes nothing that anyone experiences.

Interview question

Q: Sixty engineers, a 20-minute merge pipeline, serial merges. Tell me whether this works, and what you would change in what order.

What a strong answer covers: doing the arithmetic to 24 or 72 merges a day against demand of 120; identifying serialisation rather than duration as the binding constraint, and showing that halving the pipeline is insufficient; optimistic batching with bisection and its flip condition at a high failure rate; change-aware test selection as the largest single win; moving broad checks to post-merge detectors with a revert rule; and naming the behavioural symptom that indicates the ceiling has already been crossed.

Quick check

Quiz: Why does halving pipeline duration fail to solve a saturated merge queue at 60 engineers? Because it doubles throughput from 24 to 48 merges a day against demand of 120; the serial merge is the constraint, not the run time.

Flashcard: When does optimistic batching stop being worth it? — At roughly a 30% batch failure rate, where bisection runs dominate and serial merging is faster, which is why flaky tests must be fixed before batching is introduced.