advanced 2 min answer

A cost-sensitive platform must choose between a highly available architecture and a cheaper one with a wider failure domain. How should the decision be made?

physics-wallahcostreliabilityslobusiness-impact
Show the full answer Hide the answer

The inputs the decision needs

  • The business cost of the outage, per hour, for each affected journey. Not a general sense that downtime is bad — a number, even a rough one, because everything else is compared against it.
  • The probability and duration of the failure the redundancy prevents, from actual history rather than from vendor availability claims. Most organisations' outages are deployments and configuration rather than infrastructure, and multi-zone redundancy addresses neither.
  • The recurring cost of the more available design, including the operational cost — more components to monitor, more failure modes to understand, more on-call knowledge.
  • What degradation is possible. A journey that can degrade rather than fail changes the calculation entirely, and it is usually far cheaper than redundancy.

The decision that usually follows

Different answers for different journeys. For an education platform: live class delivery is the product and justifies redundancy; the content catalogue can serve stale from a cache; enrolment can queue; analytics can be down for hours with no consequence.

Applying one availability target across all of them is what makes reliability unaffordable and therefore under-delivered where it matters.

The lever that is usually cheaper than redundancy

Graceful degradation. Serving cached content, queueing writes, showing last-known-good with a timestamp, and falling back to a simpler experience are all far cheaper than duplicating infrastructure, and for many journeys they deliver a better outcome than a failover that takes minutes.

The design question is what the product can do when a dependency is unavailable, and it is a product conversation that engineering frequently has alone and therefore answers badly.

The framing for the business

"We can spend X and reduce expected downtime by Y hours per year, against an outage cost of Z per hour." That sentence makes the decision the business's rather than engineering's, which is where it belongs — and it usually reveals that the expensive option is justified for one journey and not for the other five.

For a price-sensitive market, the cost per user of the reliable design is itself a competitive consideration, and an architecture that prices the product out of its market has optimised the wrong thing.