concept

Fault Tolerance

Continuing to operate correctly despite the failure of some components, by design rather than by luck.

resilienceredundancyfailure

Worth separating three words that get used interchangeably. A fault is a component deviating from spec. An error is the resulting incorrect internal state. A failure is the system delivering the wrong service. Fault tolerance is the work of stopping faults becoming failures.

The mechanisms are: redundancy (another instance can take over), isolation (a fault is confined — bulkheads, cells, zones), detection (health checks, timeouts, heartbeats), and recovery (retry, failover, restart, degrade).

The part usually missing is detection latency. A design with perfect failover and a 60-second detection window has a 60-second outage per failure, and that number goes straight into the availability arithmetic. How fast you notice is as much a design parameter as what you do about it.