An automated tool recommends reducing your instances by 60% based on average CPU utilisation. What do you check before accepting?
Show the full answer Hide the answer
What is being tested
Whether you know that rightsizing has constraints beyond utilisation, and that naive application causes outages.
What to check
1. Peak, not average. Average utilisation is the wrong input entirely. Look at percentiles over at least two weeks covering the weekly cycle and any known peak. A workload averaging 20% may hit 85% every weekday morning.
2. Failure-domain headroom. With three availability zones and a requirement to survive losing one, maximum utilisation is (N−1)/N — about 66%. Sizing to 90% means a zone failure takes the service down. The tool does not know this requirement exists.
3. Latency headroom. Queueing delay rises non-linearly near saturation: roughly 4x the service time at 80% utilisation, 9x at 90%. For a latency-sensitive service, 50–70% at peak is the target, and "efficient" utilisation destroys the SLO.
4. Which dimension is actually constrained. The instance may be sized for memory while CPU sits idle. A recommendation based on CPU alone will cause an out-of-memory failure. Check memory, network, disk I/O and connection limits.
5. Burst behaviour. Some instance types have burstable CPU with credits. Downsizing may work perfectly until credits are exhausted, which is a delayed and confusing failure.
6. Growth. Sized for today, exhausted in a quarter, and re-sizing has its own operational cost.
7. What the workload does that is not steady-state. Startup, cache warming, batch windows, deployment surges, backup jobs.
How to apply it safely
- Not in bulk. Change a subset, monitor for a full weekly cycle, then continue.
- Non-production first.
- Watch latency percentiles, not just error rates. The first symptom of insufficient capacity is a rising p99, well before anything errors.
- Keep the ability to revert quickly, and know how long reverting takes.
Where the recommendation is probably right
Non-production environments sized identically to production for no reason; databases provisioned for a peak that never arrived; container resource requests set generously and never revisited (which also prevents the scheduler from packing efficiently); storage volumes at ten times used capacity.
What a strong answer adds
That rightsizing must be repeated. Drift returns within a year as workloads change, so a one-off exercise captures a saving and then loses it. Continuous recommendation with human review of the headroom constraints is the sustainable form — and the headroom rules should be encoded so the tool stops recommending things that would break the availability requirement.