Model Evaluation Service  ·  View 05 of 21  ·  People and journeys

Journey — Chasing a Regression Users Already Met

The gate missed something. What the quality lead does in the next hour, and what the platform owes them.

Editable source SVG draw.io All views
Quality Lead owns 18 suites Goal — Explain how a regression got past the gate, and make sure that class never does again Trigger — A spike in regeneration rate on the canary cohort, four hours after rollout Done when — Rolled back, root cause named, and a permanent regression test in place 1 · Notice online signal 2 · Confirm ◆ moment of truth 3 · Roll back ≤ 5 min ◆ moment of truth 4 · Explain 5 · Prevent What they do Sees the guardrail alert Checks for a confound Reads canary deltas Calls the rollback Finds the offline blind spot Turns failures into a suite Where Pager Confound register Deployment pipeline Slice report Policy repository What the platform does Guardrail breach fires Names overlapping changes Auto-rollback trigger Offline-online correlation Redacts · versions · gates Feeling Alarmed Uncertain — was it us? Relieved Exposed — gate missed it Rebuilt trust User Journey — Quality Lead Chases a Regression Users Already Met The worst moment is not the regression — it is the twenty minutes spent not knowing whether the model or a concurrent UI change caused it. The confound register is a structural answer to that moment, not a reporting nicety. v 1.0 · owner Data & AI Global Practice · date 2026-09

The trough

  • The worst moment is not the regression — it is twenty minutes of not knowing whether the model or a concurrent UI change caused it.
  • The confound register is the structural answer to that moment. Without it, attribution is a guess made under pressure.

Decisions it forces

  • Automatic rollback on a guardrail breach, without waiting for a scheduled analysis (≤ 5 min p99).
  • Every production failure becomes a permanent targeted regression example, or the same class ships again.

Risks

  • A suite that has never predicted an online outcome is cost, not coverage — which is why offline-online correlation is measured per suite.
◀  Journey — Taking a Candidate Through the Gate All views Layered Architecture  ▶
Solution Architecture v1.0 · Google Cloud · Data & AI Global Practice · 2026-09 · 21 views · 18 architecture decision records05-journey-regression-hunt