Model Evaluation Service · View 12 of 21 · Runtime
Decisions
- When the incumbent has no score in the resolved frame, the run re-baselines it first. That is the normal consequence of changing a dataset, rubric or judge — not an exception.
- Judge scores are written asynchronously and append-only; the gate reads them, it is never handed them.
- The harness error rate is persisted with the run, because it decides whether the run may yield a PASS at all.
Targets
- 6,500 examples × 3 generations ≤ 90 min p95, ≤ 150 min p99 (stated assumptions).
- Re-baseline across all gating suites ≤ 12 h, so a frame change lands inside one working day.
Risks
- The re-baseline step is where a frame change becomes visibly expensive. If it is not budgeted, it will be skipped — and skipping it is how the moving-baseline problem returns.
