Distributed Workflow Orchestration Platform  ·  View 26 of 31  ·  Operations

Operations Lifecycle Loop

The loop an operator actually works in, and the edge that makes it a lifecycle rather than an escalation path.

Editable source SVG draw.io All views
Observe
SLO · queue age · DLQ
Observe...
Detect
burn-rate alert fires
Detect...
Triage
timeline + trace + payload
Triage...
Act
retry · replay · cancel · scale
Act...
Recover
verify SLO restored
Recover...
Learn
postmortem · chaos test
Learn...
Running platform
Running platform
burn rate
burn rate
page
page
root cause
root cause
intervention
intervention
green
green
new guardrail
new guardrail
Operations Lifecycle — The Loop That Has to Close
Operations Lifecycle — The Loop That Has to Close
Security / platform
Security / platform
Person or role
Person or role
Application we own
Application we own
The loop only closes at Learn: every incident produces either a chaos experiment that reproduces it or a policy that prevents it. Without that edge this is an escalation path, not a lifecycle.
The loop only closes at Learn: every incident produces either a chaos experiment that reproduces it or a policy that prevents it. Without that edge this is an escalation path, not a lifecycle.
v 1.0 · owner Data & AI Global Practice · date 2026-08
v 1.0 · owner Data & AI Global Practice · date 2026-08
Text is not SVG - cannot display

Decisions

  • The loop only closes at Learn. Every incident produces either a chaos experiment that reproduces it or a policy that prevents it; without that edge this diagram would be an escalation path with a curve drawn on it.
  • Act is bounded to four reversible operations plus one irreversible: retry, replay, cancel and scale are safe; terminate is not, and it demands a reason and fires compensation.
  • Triage starts from the execution timeline rather than from logs. The event log in view 12 is the operator's primary tool, which is a large part of why it exists.

What operations owns

  • The Operator API surface: get execution, cancel, pause, resume, retry task, terminate task, get history, list dead-letters. Every one is audited with an actor and a reason.
  • Pause and resume operate at execution level and take effect at the next dispatch decision. In-flight tasks are allowed to complete rather than being abandoned, which keeps pause cheap and safe.
  • Runbooks are versioned with the code they describe, and every failure mode in view 31 has one.

Risks

  • Operator actions are powerful enough to cause data loss. Roles are separated between read, intervene and destroy, with the destructive role behind PIM and time-bound (view 27).
  • A quiet loop is not evidence of health. Game days and chaos experiments in staging are what keep the path exercised between real incidents.
  • Learn is the step organisations quietly drop under delivery pressure. It is drawn as a required edge so that dropping it is a visible decision.