Chaos Engineering Platform  ·  View 17 of 21  ·  Operations

Release Pipeline

Shipping the platform that breaks production, gated on proving reversion rather than on passing tests.

Editable source SVG draw.io All views
Source Repo + review 2 approvals Adapter contracts Build Cloud Build SLSA provenance Artifact Registry signed images Adapter gate Revert proof suite every adapter Lease expiry test kill the issuer Staging Staging cluster Self-chaos suite platform as target Canary One cluster, one class 24 h Abort-time SLO check p99 ≤ 10 s Fleet Cluster-by-cluster 6 waves Automatic rollback on merge contract tests signed image under test then expiry clears platform as target green measured in budget over budget Release Pipeline — Shipping the Platform That Breaks Production Data store Application we own Security / platform Decision point synchronous failure / alternate An agent release is gated on proving reversion, not on passing unit tests. That gate is the product. v 1.0 · owner Reliability Architecture · date 2026-09

Decisions

  • An agent release is gated on the revert-proof suite and a lease-expiry test that kills the issuer and confirms faults clear. That gate is the product.
  • The canary measures abort latency against its SLO and rolls back automatically when it is over budget. Correct behaviour under abort is a release criterion, not a dashboard.
  • The platform is a target of its own suite in staging before any fleet rollout.

Assumptions

  • One cluster, one fault class, 24 hours of canary before a six-wave fleet rollout.
  • Abort latency p99 ≤ 10 s measured on the canary, not asserted from staging.

Risks

  • A revert-proof suite can only prove reversion for the faults it knows how to create. A new adapter is the highest-risk change this platform can ship.
◀  Deployment Architecture All views Observability  ▶
Solution Architecture v1.0 · Google Cloud · Reliability Architecture · 2026-09 · 21 views · 16 architecture decision records17-release-pipeline