AI Code Completion Service  ·  View 16 of 21  ·  Runtime

Model Promotion and Gating

An offline gate that can only refuse, and an online verdict that can only be read a day later.

Editable source SVG draw.io All views
Register Weights + config content addressed Sign & admit Key Vault Offline gate Held-out contexts licensed only Eval harness Promotion allowed? Canary 1% arm stable assignment Control arm held back Cache warm-up before measuring Online verdict Acceptance rate immediate, noisy Retention at 120 s canonical TTFT p95 Decide Regression > 1.5%? Ramp or revert Staged ramp by region Alias to previous ≤ 10 min Previous stays mounted 72 h allowed then measure no yes Model Promotion — Offline Gate, Online Verdict Data store Security / platform Application we own Decision point Opportunity synchronous failure / alternate The offline suite is a gate, not a verdict: it can refuse a candidate before it reaches a canary, and it can never promote one. Only retention can. v 1.0 · owner Developer Experience Architecture · date 2026-10

Decisions

  • Retention at 120 s is canonical; acceptance rate is a fast, noisy companion and TTFT is a guard rail. A model that wins acceptance and loses retention is refused.
  • Warm-up precedes measurement, or the candidate arm is measured on cold-start latency and judged on an artefact of its own newness.
  • The previous version stays mounted for 72 hours; rollback is an alias move and completes in ≤ 10 minutes.

Assumptions

  • Canary ≥ 1% for ≥ 24 h; block on relative acceptance regression > 1.5% at 95% confidence or any retention regression.
  • Model artifacts are content-addressed and signed; an unevaluated model cannot be promoted.

Risks

  • A 24-hour canary at 1% is 24 million requests and still underpowered for small effects in a single language. Per-language verdicts need either longer windows or a larger arm, and both cost.
◀  Four Fates of One Keystroke All views Deployment Topology  ▶
Solution Architecture v1.0 · Microsoft Azure · Developer Experience Architecture · 2026-10 · 21 views · 15 architecture decision records16-model-promotion-and-gating