Model Evaluation Service  ·  View 19 of 21  ·  Operations

The Evidence Loop

Six steps from a submitted candidate back into the corpus — and the one step most platforms skip.

Editable source SVG draw.io All views
Submit config pinned Measure frame-bound Decide per slice Expose shadow → canary Observe guardrails Capture failures → suites Evaluation 25 / week candidate id scores + evidence verdict live traffic online outcome new regression set The Evidence Loop — From Candidate to Corpus and Back Application we own Decision point Security / platform Data store The loop closes only if step 6 actually happens. A production failure that does not become an example is a regression the gate will miss again. v 1.0 · owner Data & AI Global Practice · date 2026-09

What closes the loop

  • Step 6 is the one that is skipped. A production failure that does not become an example is a regression the gate will miss again.
  • Capture is automated and privacy-gated, because a manual step at the end of an incident is a step that does not happen.

Assumptions

  • 25 candidates per week through the loop, 6 of which reach a full 42,000-example regression sweep (stated assumptions).

Open

  • Whether online evidence gates the full rollout or only observes it is Core Architecture Question 4. This view assumes staged exposure with offline gating entry to canary.
◀  Observability and Operations All views Security Trust Zones  ▶
Solution Architecture v1.0 · Google Cloud · Data & AI Global Practice · 2026-09 · 21 views · 18 architecture decision records19-candidate-lifecycle