Compute Optimisation

Definition

Compute is usually the largest or second-largest line on a cloud bill, and it responds to four distinct classes of intervention that are frequently confused with one another.

The four levers

1. Right-sizing. Match instance size to measured utilisation. Fast, low risk, and typically removes 30–50% in an estate that has never done it. Bounded — you can only do it once, and drift returns it.

2. Scheduling. Non-production environments running nights and weekends are pure waste. Scheduled shutdown often removes 60–70% of non-production spend and is a configuration change. Batch workloads moved to off-peak windows may also unlock cheaper capacity.

3. Purchasing model. Commitments (reserved capacity, savings plans) for the stable base; spot or preemptible capacity for anything interruption-tolerant, at 60–90% less. Spot is the largest single discount available and it is an architectural property — the workload must handle being terminated with short notice, which means checkpointing, graceful shutdown and idempotent work units.

4. Architecture. The largest and slowest lever: doing less work. Caching, removing N+1 patterns, incremental processing instead of full recomputation, a more efficient runtime, better algorithms. Continuous profiling makes this tractable by answering "which code path is responsible for 8% of our compute bill" — a question with a direct financial answer that frequently unlocks engineering time a latency argument cannot.

The autoscaling nuances

• Scale on the right signal. CPU is a proxy; queue depth, request concurrency or a business metric is usually better and reacts sooner.
• Scale-in must be graceful, or you drop in-flight work on every scale event.
• Scaling has lag. Minutes, which does not cover a step change — so keep enough headroom to survive the ramp.
• Do not scale to zero for latency-sensitive workloads unless cold start is acceptable at p99.

Failure scenarios

• Committed before right-sizing, locking in the wrong sizes for a year.
• Spot adopted without handling interruption, producing data loss or failed jobs.
• Autoscaling on CPU for a workload bounded by I/O or by a downstream, so it never scales when it should.
• Over-provisioned "for safety" at every layer, compounding multiplicatively.
• Optimising compute while the bottleneck is elsewhere, so nothing improves and the bill falls only slightly.

Interview question

"Which compute cost lever has the largest payoff and why is it the one teams attempt last?"