A nine-engineer team runs one deployable. Checkout holds a p99 target of 200 ms; a PDF-generation path in the same process takes 2-40 seconds and saturates CPU during the morning peak, which has caused three checkout incidents this quarter. The team has no platform engineers. What is the smallest change that fixes it?

The deciding property

The problem is resource contention between two workloads with incompatible latency profiles, in one process that shares a CPU allocation and a thread pool. The property the team needs is scaling locality for exactly one path. Nothing else about the system is broken.

A workload whose service time varies by a factor of 20 cannot share a scheduler with a workload on a 200 ms budget. Queueing theory says why: as utilisation rises, waiting time grows with the variance of service time as well as its mean. A few 40-second units of work are enough to put 200 ms requests behind them, and the symptom is exactly what the team sees — p99 collapse while average CPU looks acceptable.

Why extracting one path

It fixes the contention and nothing else. The PDF worker gets its own CPU allocation, its own scaling rule (queue depth, not request rate) and its own failure domain. Checkout keeps its process, its deployment and its simplicity. Nine engineers now own two deployables instead of one, which is an operational increase they can absorb; they could not absorb eight.

The queue earns its place here rather than being fashion: the work is already asynchronous from the user's point of view (a PDF that takes 40 seconds is not awaited in a request), so the queue removes the timeout problem instead of creating a new one. It also gives the one planning number the team needs: age of the oldest unprocessed job, which is the signal to scale and the signal to alert on. Queue length is not, because length says nothing about whether anyone is waiting.

Why the other options fail

Split into per-domain microservices. This is the right answer at a different scale. For nine engineers it converts one deploy into eight, adds network calls to paths that were function calls, and introduces distributed failure modes to fix a CPU-sharing problem. The cost is paid on every future change; the benefit is available from one extraction.

Larger instances. This is the tempting local fix, and it moves the threshold without removing the mechanism. Doubling CPU doubles the number of concurrent PDFs needed to hurt checkout, so the incident returns when volume grows or when someone uploads a 400-page document. It also pays for peak CPU on every instance for the whole day.

Background threads in the same process. It makes the contention harder to see without reducing it. The threads share the same CPU quota, the same memory limit and the same garbage collector; a burst still starves the request path, and now the request path and the batch path die together on one out-of-memory kill. The only version of this that helps is a hard CPU cgroup for the background work, which is most of the operational cost of a second deployable with none of its scaling independence.

When this is the wrong answer, and what would flip it

If the PDF path is the only expensive thing the system will ever do and volume is flat, the concurrency limit alone is enough and the second deployable is overhead.