Spot Capacity Orchestrator  ·  View 22 of 32  ·  5 · Runtime

Fallback and Return

Three triggers, a floor that is restored without asking, spend inside a declared authority, escalation beyond it, and a return slow enough not to oscillate.

Editable source SVG draw.io All views
Trigger Interruption breach above declared rate Shortfall ≥ 5 min unmet Deadline risk ETA past deadline Floor Below floor? Restore floor committed first · no approval Authority Within 24 h cap? Staggered fallback priority · 40 per min Beyond authority Page budget owner 15 min to answer Record shortfall not spend Return Healthy for 60 min? Return 20% per hour newest fallback first yes yes no no answer yes Fallback and Return — Floor First, Spend Within Authority, Return Slowly Risk / gap Decision point Application we own Interface / broker synchronous failure / alternate All three triggers enter at the floor check. Every active fallback is tested for return each cycle, and records its trigger, duration and incremental cost. v 1.0 · owner Compute Platform Architecture · date 2026-09

Decisions

  • Restoring the floor is pre-authorised by the tier and draws first on committed capacity that is already paid for, so it is automatic and nearly free at the margin.
  • Fallback above the floor is automatic up to the contract's 24-hour spend cap. Beyond it the arbiter pages the named budget owner and records shortfall if nobody answers in 15 minutes. A cost incident is allowed to become an availability incident, visibly, rather than an unbounded bill at 03:00.
  • Fallback is staggered at 40 instances a minute per region in priority order: deadline risk, then tier. Every workload reaching for guaranteed capacity at once is a demand spike on the capacity every guaranteed tenant also wants.

Return

  • Pools must be healthy for 60 minutes and below half the workload's declared interruption rate. Return then moves 20% an hour, newest fallback first. Hysteresis costs a little spend and prevents the fleet oscillating between purchase models.

Measured

  • Each fallback records trigger, duration, instances and incremental cost against the workload, so the price of reliability is attributed rather than absorbed.
◀  Correlated Reclamation All views Control-Plane Outage  ▶
Solution Architecture v1.0 · Compute Platform Architecture · 2026-09 · 32 views · open source, fully on-premises22-fallback-and-return