Enterprise Generative Search — Azure and Open Source  ·  View 31 of 41  ·  Operations

CI/CD and Index Migration

Code, prompts, index schemas and models through one path with the same gates.

Editable source SVG draw.io All views
Source Application code GitHub, trunk-based Prompt registry versioned, reviewed Index schema declared, migrated Golden set 1,400 queries Build Container build GitHub Actions SBOM and signature Syft and Cosign Unit and contract tests retriever contract Evaluate Retrieval metrics Recall@50, nDCG@10 Generation metrics groundedness, citations Adversarial suite injection and leakage Cost per query budget regression Stage Shadow index build new alias, not swapped Shadow traffic 10% mirrored, not served Side-by-side scoring same queries, both stacks Gate Thresholds met no metric below baseline Human approval prompt and policy changes Change record what moved and why Release Argo CD sync GitOps, per environment Canary 5% 30 minutes Alias swap index cutover is atomic Watch 48-hour watch online quality signals Auto rollback alias back, pods back Baseline updated new reference scores revert CI/CD — Shipping Code, Prompts, Index Schemas and Models Through the Same Gates Application we own Data store Security / platform Decision point failure / alternate A prompt change and a code change take the same path. Treating prompts as configuration is how a quality regression reaches production without a review. v 1.0 · owner Data and AI Global Practice

Decisions

  • A prompt change takes the same path as a code change. Treating prompts as configuration is how a quality regression reaches production with no review and no rollback.
  • An index schema or chunking change is a shadow build with an atomic alias swap, never an in-place migration. Two embedding versions must never answer the same query.
  • Quality and cost gates block the release. A change that improves groundedness and doubles the bill is a decision for a human, and the pipeline makes it one.

Numbers

  • Pull-request evaluation runs on a 40,000-document fixture corpus in about 9 minutes; nightly runs the full golden set against the full corpus.
  • Canary at 5% for 30 minutes, then a 48-hour watch with automatic rollback on the online quality signals in view 34.
  • Shadow index build for a full re-chunk: 14 hours, run against production traffic mirrored at 10% before any swap.

Risks

  • Shadow builds double the index cost for their duration. That is budgeted per migration rather than provisioned permanently.
  • Evaluation gates can be gamed by a golden set that drifts towards what the system is good at. The set grows from real failures via view 33, and its composition is reviewed quarterly.
◀  Deployment Architecture All views Evaluation Harness  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 41 views · Microsoft Azure and open source31-cicd-and-index-migration