API Gateway Platform  ·  View 17 of 21  ·  Operations

Configuration Rollout and Canary

A routing change and an upstream canary are the same artefact, so they use the same pipeline.

Editable source SVG draw.io All views
Author Change in Git routes, policies Review + approval second pair of eyes Validate Schema + policy lint CI Replay against samples routing diff Commit Config version N immutable, signed Snapshot published N and N-1 retained Canary fleet 1 pod per region pulls N Score 5 min error, p99, 5xx Canary traffic 1% → 5% → 25% sticky per credential Rolling verdict halt on breach Fleet 100% of pods on N 99% in 10 s Version coverage reported per pod Revert path Revert to N-1 ≤ 30 s, one action Auto-revert ≤ 90 s of breach breach pin N-1 Configuration Rollout and Canary Application we own Decision point Data store Security / platform Risk / gap failure / alternate The same pipeline promotes a routing change and an upstream canary, because both are the same artefact: a configuration version. v 1.0 · owner Integration Platform Architecture · date 2026-09

Decisions

  • Configuration is reviewed in Git and validated by replaying sample requests against the new routing table, so a routing diff is visible before it is live.
  • The canary runs twice: first on a small set of pods carrying normal traffic, then as a weighted split of traffic across upstream variants. The first catches a bad config, the second catches a bad service.
  • N−1 is retained as a published snapshot, so revert is selecting a version rather than re-running a pipeline.

Numbers

  • Canary weights 1% → 5% → 25% → 100%; automatic revert within 90 s of a breaching signal; operator revert to last known good in ≤ 30 s (assumptions).
  • Version coverage reported per pod, ≥ 99% within 10 s.

Risks

  • At 1% weight on a low-traffic route, the canary window may contain too few requests to distinguish a regression from noise. The rollout controller must be able to say "insufficient evidence" rather than "healthy" (ADR-13).
◀  Deployment Architecture All views Observability  ▶
Solution Architecture v1.0 · Google Cloud · Integration Platform Architecture · 2026-09 · 21 views · 18 architecture decision records17-config-rollout-and-canary