Chaos Engineering Platform  ·  View 13 of 21  ·  Runtime

Abort and Degradation Paths

Four triggers, two decisions, one mechanism — and every path on the page ends with the fault gone.

Editable source SVG draw.io All views
Trigger SLI breach Telemetry blind Control plane or link lost Operator or incident Decision Withhold renewal Kill switch Mechanism Lease expires ≤ 15 s Watchdog reverts Confirm Re-read target state Settling check Outcome Fault cleared Revert unconfirmed INCONCLUSIVE Escalation Out-of-band reverter Suspend fault class abort rule no signal nothing renews stop TTL out fleet-wide revert re-read gone present baseline no coverage escalate class off Abort and Degradation Paths Risk / gap Person or role Decision point Security / platform Application we own synchronous failure / alternate Every red path ends with the fault gone. There is no path on this page where uncertainty keeps a fault alive. v 1.0 · owner Reliability Architecture · date 2026-09

Decisions

  • There is no path on this page where uncertainty keeps a fault alive. Losing the control plane, losing the link, and losing telemetry all converge on the same expiry.
  • An unconfirmed revert is an incident, not a metric: the out-of-band reverter is invoked, on-call is paged, and that fault class is suspended fleet-wide pending review.
  • Telemetry blindness produces INCONCLUSIVE. Absence of a signal is never recorded as health.

Assumptions

  • Lease TTL 15 s is the worst-case time an unattended fault survives, and the global kill switch halts injection fleet-wide within 30 s.
  • One unconfirmed revert per ~900 injected faults.

Accepted clutter

  • One route-check warning remains: two trigger labels converging on the same decision box sit close. Cutting either label would remove the reason the path exists.
◀  Critical Flow — One Experiment, Start to Verdict All views Guardrail Evaluation  ▶
Solution Architecture v1.0 · Google Cloud · Reliability Architecture · 2026-09 · 21 views · 16 architecture decision records13-abort-and-degradation-paths