AI Executive Office — CXO Assistant Platform  ·  View 22 of 30  ·  6 · Operations

Delivery — Code and Configuration

Two pipelines at different speeds, and the gate that stops a prompt change quietly degrading every tenant's answers.

Editable source SVG draw.io All views
Source Product repo app · Bicep Prompt packs versioned Tenant config repo one branch per tenant Industry packs gov · bank · energy Build Container build signed · SBOM Bicep what-if plan reviewed Config schema check reject unknown keys Quality gates Unit and contract Scan code · image · IaC Golden-question eval accuracy · grounding Red-team suite injection · leakage Tenant isolation test cross-tenant probes Environments Dev synthetic data Staging synthetic tenant Ring 1 one pilot tenant Ring 2 all tenants Release control Feature flags per tenant Model pinning tenant opts in to upgrades Tenant change window sovereign tenants After release Watch the eval metrics 24 h Roll back image + prompt together regression blocks grounding drop Delivery — Two Pipelines, Code and Tenant Configuration Application we own Data store Decision point Security / platform Interface / broker failure / alternate A prompt is a release artefact and moves through the same gates as code. Tenant configuration travels on its own faster pipeline, but it is still schema-checked, versioned and reversible. v 1.0 · owner Data & AI Global Practice · date 2026-09

Decisions

  • A prompt is a release artefact. It is versioned, evaluated and rolled back together with the container image, because a prompt change can break accuracy exactly as thoroughly as a code change and is far easier to make casually
  • Tenant configuration travels on its own faster pipeline, but it is schema-checked, versioned and reversible. "Configuration, not custom code" only holds if the configuration is engineered
  • Ring deployment: one pilot tenant before all tenants. Sovereign tenants have their own change windows and may pin a model version

The gates that matter

  • Golden-question evaluation with a no-regression rule, per tenant where the tenant has supplied questions
  • Cross-tenant isolation probes on every release — the pooled tier's central control, automated because review cannot prove a negative
  • Red-team suite for prompt injection and data leakage, run against the retrieval path with adversarial documents

Risks

  • Evaluation sets rot. A suite that passes for six months usually means it stopped being representative, so failures mined from production are fed back into it continuously (view 24)
◀  Deployment Topology All views Observability  ▶
Solution Architecture v1.0 · Azure-native, multi-tenant, sovereign-capable · Data & AI Global Practice · 2026-0922-cicd-and-tenant-config