Autoscaling Controller  ·  View 32 of 37  ·  5 · Operations

The Operator's Loop

Alert, ask the journal, buy time, propose a policy, prove it, promote or discard.

Editable source SVG draw.io All views
Alert Fires named condition Ask the Journal why this capacity Override expires in 4 h Draft New Policy as a version Replay Counterfactual against history Promote or Discard cost diff shown Operate tune against evidence a target, a time the driving signal capacity held candidate version what it would have done policy or nothing The Operator's Loop Security / platform Data store Application we own Decision point An override buys time to think; it never becomes the fix. Expiry is mandatory, so control returns to the policy whether or not anyone remembers. v 1.0 · owner Data & AI Global Practice · date 2026-09

Decisions

  • An override buys time to think and never becomes the fix. Expiry is mandatory at 4 hours by default.
  • The fix is a new policy version, proved by counterfactual replay against the history that produced the alert.
  • Discarding is a legitimate outcome of this loop and is recorded like any other.

Why expiry is mandatory

  • Every long-lived autoscaler accumulates permanent manual pins that nobody remembers the reason for, and they are the largest single source of unexplained cost.
  • Control returns to the policy automatically whether or not anyone remembers.

Assumptions

  • 4 hours is a stated assumption chosen to outlast an incident and not a shift. Extending an override is a new override with a new record.
◀  Cost and the Price of Stability All views Trust Zones  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 37 views · Azure and open source32-operations-loop