Capacity Insurance

Cost programmes look for resources that are not being used. Three categories are not being used and must not be removed: headroom that absorbs load spikes, redundancy that absorbs failures, and failure-domain capacity that absorbs the loss of a zone or a region.

These look identical to genuine waste on any utilisation dashboard, which is why cost programmes cause incidents. The distinction is not visible in the metric; it is visible only in the answer to a question about purpose.

The distinguishing test

Under what circumstance does this become valuable, and how likely is that circumstance?

• Unattached storage volumes: no circumstance. Waste.
• A non-production environment running overnight: no circumstance. Waste.
• 30% CPU headroom: valuable during every daily peak and every instance failure. Insurance.
• An idle standby region: valuable during a regional failure. Insurance.
• Telemetry covering a failure mode that occurs annually: valuable when it occurs. Insurance.

If there is no answer, it is waste. If the answer is "when something fails", it is insurance, and its removal is a reliability decision rather than a cost decision.

Why it must be defended explicitly

Because utilisation is the number leadership watches, and "we run at 65% on purpose" needs an argument. Two arguments are available and both should be made:

The queueing argument. Delay grows roughly as 1 / (1 − utilisation) — a factor of 3.3 at 70%, 10 at 90%, 20 at 95%. Latency does not degrade gradually toward saturation; it goes asymptotic. The headroom is what keeps latency bounded, so the utilisation target is derived from the latency SLO, not from cost efficiency.

The N-1 argument. If steady-state load requires all three zones, losing one leaves 67% of required capacity and the survivors fail. Surviving a zone failure means running at roughly two-thirds utilisation. This is the cost most often cut in budget review, and the reason zone redundancy so frequently does not work when it is needed.

Implementation patterns

• State headroom as a design parameter on capacity dashboards, alongside utilisation, so it reads as intent rather than as slack.
• A reliability floor stated as policy that cost optimisation cannot cross — headroom for failure domains, redundancy, and rehearsed degradation are out of scope for cost review.
• Separate the peak-event budget from the baseline budget, with automatic expiry, so peak capacity is an approved time-bounded spend rather than a permanent overrun.
• Reduce the failure domain if the headroom is unaffordable — smaller cells, more of them — rather than assuming the failure will be gentle.

Industry example

The recurring pattern is a cost programme that reduces instance sizes based on average utilisation and is followed by incidents. The method omitted percentiles, failure headroom and the binding resource, and it removed the buffer that made routine events routine.

The same dynamic appears in peak retail events, where efficiency targets and peak readiness pull in opposite directions all year. Governance that ignores the tension produces either an efficient platform that fails at peak or a peak-ready platform whose cost nobody can defend — which is why the two budgets must be separated and the reversal automated.

Failure scenarios

• Utilisation optimised as a goal, driving the system into the non-linear latency region.
• Peak capacity that becomes permanent because reclamation depended on someone remembering.
• Redundancy cut to improve a team's chargeback number, with the incident cost borne by the business.
• Headroom removed fleet-wide in one sweep, so the resulting degradation has no single attributable cause.

Trade-offs

Insurance is a real, continuous cost against an event that may not occur this year, and it is legitimate to ask whether the premium matches the risk. That question should be answered with the impact arithmetic — what does an hour of unavailability cost this system — rather than with a utilisation target.

Where the arithmetic does not justify the headroom, the honest response is to lower the availability target explicitly rather than to keep the target and quietly remove the capacity that delivers it.

Interview question

"Finance has identified 30% idle capacity across your fleet and wants it removed. Walk me through which of it you would give up, which you would defend, and what evidence you would bring."