AI Code Completion Service · View 05 of 21 · People and journeys
The trough is Judge
- A more confident model raises acceptance while lowering retention: developers take the suggestion and delete it twenty minutes later. Acceptance alone cannot see this.
- Retention arrives a day late, which means the canary window is set by the metric's latency rather than by the traffic needed for significance.
Decisions
- The offline suite is a gate that can refuse and never promote: correlation with retention is too weak to be a verdict.
- Canary measurement starts only after prefix caches have warmed, or the new arm is measured on cold-start latency and looks worse than it is.
- The previous model stays mounted for 72 hours so rollback is an alias move rather than a redeploy.
Assumptions
- Canary ≥ 1% of traffic for ≥ 24 h; rollout blocked on a relative acceptance regression > 1.5% at 95% confidence, or any retention regression.
- Rollback to the previous registered version within 10 minutes.
