CI/CD for ML
Testing pipelines that emit models, data-dependent tests, progressive delivery and automated retraining.
6concepts
69flashcards
44minutes of reading
- 01 Automated Retraining and Its Triggers When a retraining loop is worth building, the four trigger designs and what each optimises, and the failure modes that make automated retraining actively dangerous.
- 02 Data-Dependent Tests and Behavioural Suites Why aggregate metrics are a poor gate, how curated behavioural cases catch regressions that averages hide, and the discipline of turning every production failure into a permanent test.
- 03 LLM Regression Testing in CI How to gate pull requests on a stochastic system, covering golden-set sizing, paired comparisons, judge variance, flaky thresholds and the per-PR cost budget that decides what runs where.
- 04 Progressive Delivery for Models The staged rollout patterns that separate correctness validation from quality validation, why shadow deployment answers a question canary cannot, and what each stage should be watching.