A team is asked to reduce infrastructure cost by 30%. How do you make the reliability trade-off explicit rather than accidental, and what should be cut first?
Show the full answer Hide the answer
The framing that prevents a bad outcome
A cost reduction target with no reliability constraint attached guarantees that reliability will be spent without anyone deciding to spend it. The right first move is to establish what is being purchased with the current spend and what the business is willing to give up.
The productive question is not "where can we cut?" but "what reliability are we currently buying, do we need all of it, and what would we observe if we had less?"
Make it explicit with the error budget
If SLOs exist, this is answerable with data. A consistently unspent error budget is evidence of over-investment in reliability, purchased with money and with velocity. That is the strongest possible justification for reducing redundancy: the system is more reliable than the business asked for.
If SLOs do not exist, establishing them is the prerequisite, because otherwise every cut is a bet against an unknown requirement.
What to cut, in order of safety
- Waste with no reliability content: unused resources, orphaned volumes, oversized instances, idle environments, forgotten test infrastructure, over-retained backups and logs. This is typically 15–30% of spend in an unmanaged estate and costs nothing in reliability. Exhaust it entirely before anything else.
- Rightsizing to actual utilisation, with headroom sized from measured peak-to-average and variance rather than from habit.
- Commitment-based discounts — reserved capacity for the steady baseline — which reduce cost with zero architectural change. The only risk is committing to more than the sustained need.
- Spot or preemptible capacity for interruptible workloads: batch, CI, training, stateless serving with sufficient replicas. Large savings, and only for workloads genuinely designed to be interrupted.
- Storage tiering and retention reduction, which affects recovery capability and must be checked against actual RPO requirements rather than against assumptions.
- Non-production environment consolidation and scheduled shutdown, which affects delivery speed rather than production reliability — a real cost, and one worth naming rather than pretending it is free.
- Reduced redundancy — fewer replicas, fewer zones, a smaller standby. This is where reliability is genuinely sold, and it should be last, quantified, and explicitly approved.
Quantifying before cutting redundancy
For each proposed reduction, state: the failure it stops protecting against, the probability of that failure, the expected impact, and the annual saving. A change that saves a modest amount and increases expected annual downtime by hours is a bad trade; one that saves substantially and increases it by minutes may be excellent.
The decision belongs to the business, and the architect's job is to present it in those terms rather than either resisting the cut or making it silently.
The trap to avoid
Cutting reliability that is invisible until it is needed — the standby region, the backup verification, the spare capacity for a peak. These have no observable value on ordinary days, which makes them the easiest things to cut and the most expensive to have cut. They should be defended with a stated scenario and a number, not with a principle.