A capacity review shows services running at 85% CPU. Finance suggests raising it to 95% to save money. What is your response?

What the interviewer is testing

Whether you can explain queueing behaviour in terms a finance stakeholder can act on.

The response

Utilisation and latency are not linearly related. For a simple queueing system, waiting time is proportional to ρ/(1−ρ):

Moving from 85% to 95% roughly triples queueing delay. The saving is around 10% of compute; the cost is a large latency increase and a system that has no margin for a traffic spike.

That is the trade, expressed in a form that can be decided rather than argued.

What makes it worse than the table suggests

Variability. Bursty arrivals and variable service times push the curve up further, so a workload with a long tail of expensive requests queues badly at utilisations that would be comfortable for uniform work. Real traffic is bursty.

Failure domains. With three availability zones, losing one shifts its load onto the other two. At 85% that means 127% — an immediate overload. Running multi-zone requires staying below about 66% to survive a zone loss, which is the calculation most often missing from these conversations.

Autoscaling reaction time is minutes, so the existing capacity must absorb the spike until it catches up.

What I would propose instead

Target utilisation in the 60–70% region for latency-sensitive services, and pursue the cost saving where it does not trade against latency: rightsizing over-provisioned instances, committed capacity discounts, spot for interruptible workloads, and reducing the work itself through caching and query optimisation.

What a strong answer adds

Reframing headroom as a purchase rather than as waste: it buys failure absorption, predictable latency and spike tolerance. Naming what the business gets makes it a decision rather than an inefficiency to be eliminated.

Common weak answers

Refusing on principle without quantifying. Agreeing and relying on autoscaling.