Cost versus Reliability
Reliability is bought with redundancy and headroom, both of which cost money — so the target must be chosen rather than assumed.
Definition
Every increment of reliability has a price, and the price is non-linear: each additional nine costs substantially more than the last while delivering a smaller absolute reduction in downtime.
| Availability | Downtime per year | Typical requirement |
|---|---|---|
| 99% | 3.65 days | Single instance, business-hours support |
| 99.9% | 8.8 hours | Redundant instances, automated failover, on-call |
| 99.99% | 52 minutes | Multi-zone, tested failover, mature operations |
| 99.999% | 5 minutes | Multi-region active-active, exceptional engineering |
The step from 99.9% to 99.99% typically doubles infrastructure cost and requires a step change in operational maturity. The step to 99.999% is rarely justified outside a few domains and is frequently claimed by systems that have not measured.
How to choose the target
Not by aspiration. By the cost of unavailability:
- Revenue lost per minute of downtime, and whether it is lost or merely deferred — an e-commerce sale during a five-minute outage may simply happen five minutes later, and treating deferred revenue as lost overstates the case substantially.
- Contractual penalties and credits.
- Reputational and regulatory consequences.
- Internal cost — engineers unable to work, orders processed manually.
Set the target where the marginal cost of the next nine exceeds the marginal value. That calculation usually lands lower than instinct suggests, and stating it explicitly is what stops reliability investment being decided by whoever feels most strongly.
Per-service targets
An estate-wide target is always wrong. Checkout may warrant 99.99%; the recommendation service that degrades gracefully may warrant 99.5%, because its failure is invisible when the fallback works. Paying for uniform reliability across services with wildly different consequences of failure is one of the most common sources of over-spend.
The cheaper alternatives to more nines
Frequently better value than additional redundancy:
- Graceful degradation, so a component failure is not a service failure. Often the highest return available.
- Faster recovery. Reducing mean time to recovery improves availability arithmetically and usually costs less than preventing the failure.
- Blast radius reduction — cells or shards, so a failure affects 5% of users rather than 100%. This changes the impact of the same failure rate, which is frequently what actually matters.
Interview question
"What would it cost to move a service from 99.9% to 99.99%, and how would you decide whether it is worth it?"