Streaming Video Encoding & Packaging Pipeline  ·  View 02 of 22  ·  Context and scope

High-Level Architecture

Seven stages from delivery to publication, and the one loop in the whole design.

Editable source SVG draw.io All views
Deliver Delivery intake resumable upload / pull Master store dual-region, archive Conform Probe & conformance 126 TB/day in Remediation telecine, loudness Decide Complexity analysis shots, grain, motion Ladder solver per title Recipe store immutable, addressed Encode Lane scheduler deadline priority Batch encode fleet Spot VMs, 18%/h reclaim Stitch & verify boundary check Prove QC gate blocking Score store per rung, per window Package Segmenter & encryptor common encryption Rendition store content-addressed Publish Manifest publisher atomic swap Catalogue projection title to manifest write once chunk reads new recipe version High-Level Architecture — Master to Playable Interface / broker Data store Application we own Security / platform Decision point synchronous failure / alternate The spine is the happy path. The red return is the only loop: a failed gate revises the recipe, never retries the same one. v 1.0 · owner Media Platform Architecture

Reading the spine

  • The pale spine is the happy path and it goes one way. Every box on it is idempotent, so a replay of any stage is free.
  • The single red return is a failed gate revising the recipe. There is no retry of a failed recipe — a retry under the same parameters would produce byte-identical output and fail identically.
  • Decide comes before Encode, and that ordering is the whole economic argument: analysis is cheap per source hour, encode is not.

Why the recipe sits between

  • Separating the decision from the execution lets the same recipe be executed in batch tonight and on demand in eighteen months.
  • It is also what makes an encoder upgrade a policy change rather than a migration (ADR-04).

Scale, assumed

  • 126 TB/day of master ingest, 220,000 chunk tasks/day, 40 PB/day of delivered egress.
  • At that egress a 1% average bitrate reduction across the ladder is worth about 400 TB/day — which is why the ladder is solved per title.
◀  System Context All views Actors and Their Core Journeys  ▶
Solution Architecture v1.0 · Google Cloud · Media Platform Architecture · 2026-10 · 22 views · 15 architecture decision records02-high-level-architecture