Streaming Video Encoding & Packaging Pipeline  ·  View 17 of 22  ·  Operations

Deployment Architecture

One encode region, dual-region custody, and a standby with no fleet in it.

Editable source SVG draw.io All views
Google Cloud — primary encode region (europe-west4) Control plane — regional, multi-zone Job & lineage API Cloud Run, 3 zones Scheduler & governor GKE regional Job state Spanner regional Task dispatch Pub/Sub Batch encode fleet — zonal MIGs, Spot, no HA zone a Spot workers ~4,700 slots zone b Spot workers ~4,700 slots zone c Spot workers ~4,600 slots Latency-bound tier — reserved, multi-zone On-demand encoders GKE, warm pool Packaging origin Cloud Run Manifest cache Memorystore Regional storage Rendition bucket regional + lifecycle Chunk scratch 48 h TTL Dual-region custody (europe-west4 + europe-west1) Master bucket dual-region archive, locked Recipe store dual-region Audit log dual-region, append-only Standby encode region (europe-west1) — control plane warm, no fleet Control plane standby RTO 15 min Origin standby degraded rung set Cloud KMS multi-region Media CDN global anycast DRM licence service regenerate from custody pull-through writes segments Deployment Architecture Interface / broker Security / platform Data store Queue / topic Application we own External / third party event / async synchronous The standby region carries no encode fleet on purpose: recovery is regeneration from custody, not replication of 1.2 M rendition sets. v 1.0 · owner Media Platform Architecture

The standby deliberately has no encode fleet

  • Recovery is regeneration from custody, not replication of 1.2 million rendition sets. Masters, recipes and the audit log are dual-region; renditions are not.
  • After a region loss the published manifest degrades to the surviving rung set and refills. That is a stated, visible consequence rather than a hidden one.
  • Assumed control-plane RTO 15 minutes, RPO 5 seconds; launch-subset regeneration 30 minutes p95.

What is zonal and what is not

  • The batch fleet is zonal Spot MIGs with no availability requirement — assumed ~14,000 slots at peak across three zones.
  • Everything with a latency or correctness requirement is regional and multi-zone: the API, the scheduler, Spanner, the warm pool and the origin.

Assumption

  • Region choice (europe-west4 primary, europe-west1 standby) is illustrative. The real constraint is that custody and the encode fleet share a continent so master range reads are not a cross-ocean cost.
◀  Encoder Build Rollout All views Observability  ▶
Solution Architecture v1.0 · Google Cloud · Media Platform Architecture · 2026-10 · 22 views · 15 architecture decision records17-deployment-architecture