Performance versus Cost
The trade between latency and spend — where headroom is the mechanism and high utilisation is the false economy.
Definition
Performance is bought with capacity, redundancy, caching and proximity, all of which cost money. The trade is continuous rather than binary, and the mechanism connecting them is utilisation.
The non-linearity that governs it
Queueing delay rises sharply as utilisation approaches saturation — roughly four times the service time at 80% utilisation, nine times at 90%, nineteen times at 95%. So:
- Running at high utilisation is cheaper per unit and produces unpredictable latency.
- Running at moderate utilisation costs more and degrades gracefully.
Headroom is not waste; it is the mechanism by which latency stays predictable and spikes are absorbed. Framing it as waste is the most common way this trade is got wrong, and it is usually framed that way by someone looking at a utilisation graph without a latency graph next to it.
The levers, and where each sits on the trade
| Lever | Performance | Cost |
|---|---|---|
| More capacity | Better, reliably | Higher, linearly |
| Caching | Much better | Lower (offloads origin) — the rare win-win |
| CDN / edge | Better | Usually lower than origin egress |
| Doing less work | Better | Lower |
| Precomputation | Better read latency | Higher storage and write cost |
| Higher utilisation | Worse, non-linearly | Lower |
Note that several are not trade-offs at all. Caching, efficiency work and edge delivery improve both sides, which is why they should be exhausted before the genuine trade is made.
How to decide
Set a latency objective from a business requirement, then buy the cheapest capacity that meets it. This inverts the usual argument: instead of "how fast can we be", it is "what is fast enough, and what is the cheapest way to be that fast".
Without a stated objective, performance work is unbounded and cost work is arbitrary.
Failure scenarios
- Utilisation targeted as efficiency, destroying latency predictability.
- Optimising performance without an objective, so there is no stopping condition.
- Cutting capacity to a level that meets average load but not peak.
- Caching added without a staleness decision, trading correctness rather than cost.
Interview question
"Finance asks why your services run at 55% utilisation. What is your answer?"