Which compute cost lever has the largest payoff, and why is it the one teams attempt last?

What is being tested

Whether you can rank interventions by payoff rather than by ease, and whether you can explain the organisational reason the ranking is inverted in practice.

The ranking by payoff

1. Architecture — doing less work. Caching, removing N+1 patterns, incremental processing instead of full recomputation, better algorithms, a more efficient runtime. Unbounded upside: a workload that recomputes everything nightly when it could process a delta may be twenty times cheaper.

2. Spot and preemptible capacity. 60–90% cheaper for anything interruption-tolerant. This is an architectural property, not a purchasing decision — the workload must checkpoint, shut down gracefully and have idempotent work units. Batch, CI, data processing and stateless workers all qualify.

3. Scheduling. Non-production environments shut down nights and weekends: often 60–70% of non-production spend, achieved with a configuration change.

4. Right-sizing. 30–50% in an estate that has never done it, and bounded — you can only do it once, and drift returns it.

5. Commitments. A percentage discount on whatever you have, applied after right-sizing or you lock in the wrong sizes for a year.

The ranking by ease, which is the reverse

Commitments are a purchasing decision made by one person in an afternoon. Right-sizing is a configuration change. Scheduling is a script. Spot requires engineering work. Architecture requires engineering work, product prioritisation, and an argument for why cost work outranks feature work.

So teams do 5, 4 and 3 — capture a real and visible saving — and stop, because the remaining levers cost engineering time that has to be justified against a roadmap.

The way to unlock the architectural lever

Make cost per code path visible. Continuous profiling answers "which function is responsible for 8% of our compute bill" with a direct financial figure. That converts an abstract efficiency argument into "three days of work saves £40,000 a year", which is a prioritisation conversation product teams can actually have.

The same reframing works for caching and incremental processing: express the change in pounds per year rather than in milliseconds.

The autoscaling nuances worth mentioning

• Scale on the right signal. CPU is a proxy; queue depth or request concurrency reacts sooner and more accurately, particularly for I/O-bound workloads that never show CPU pressure.
• Scale-in must be graceful, or every scale event drops in-flight work.
• Scaling has minutes of lag, which does not cover a step change — keep headroom for the ramp.

What a strong answer adds

That optimising compute while the bottleneck is elsewhere produces very little. The order matters: find the constraint, relieve it, then re-identify — a plan that optimises three things at once usually improves one.