Finance asks why your services run at 55% utilisation rather than 90%. What is your answer?

What is being tested

Whether you can explain headroom in terms a non-engineer can evaluate, using the actual mechanism rather than an assertion.

The answer, in two parts

1. Queueing delay rises non-linearly with utilisation.

At 55% utilisation, waiting time is roughly comparable to service time. At 80% it is about four times the service time. At 90%, nine times. At 95%, nineteen times.

So moving from 55% to 90% does not make responses slightly slower — it makes them several times slower, and unpredictably so, because the tail grows fastest. That shows up directly in the customer experience and in the SLO.

This is a property of queueing, not of our code. No amount of optimisation removes it.

2. We must survive the loss of a failure domain.

We run in three availability zones and must keep serving if one fails. When one goes, the other two carry all the traffic. For that to work, peak utilisation cannot exceed about 66% — that is (N−1)/N arithmetic, not a preference.

Running at 90% means a zone failure takes the service down entirely.

The reframing

Headroom is not idle capacity. It is the mechanism by which latency stays predictable and a zone failure is survivable. Removing it is not an efficiency gain; it is buying a saving with an outage risk and a latency regression, and both have costs the finance question has not yet priced.

What to offer instead

Do not simply decline. Offer the levers that reduce cost without buying it from headroom:

• Right-sizing against measured peak — often 30–50% in an estate that has not done it.
• Non-production scheduled shutdown — frequently 60–70% of non-production spend.
• Commitments on the stable base, after right-sizing.
• Spot capacity for interruption-tolerant workloads — the largest available discount.
• Architectural efficiency — caching, removing N+1 patterns, incremental processing. These reduce the work rather than the headroom, and they improve both cost and latency.

And offer to differentiate by service. A batch workload where latency does not matter can run at much higher utilisation. A latency-sensitive user-facing service cannot. A uniform target across the estate is the actual inefficiency.

What a strong answer adds

Providing the number. "At 90% utilisation our p99 goes from 180 ms to roughly 900 ms, and a zone failure becomes a full outage. The saving is £X. Is that trade worth making?" That converts an argument into a decision the business can actually take, which is the architect's job here.