Streaming Video Encoding & Packaging Pipeline  ·  View 10 of 22  ·  Data

Data Flow — One Master Through

Seven columns, of which only the first three cannot be rebuilt.

Editable source SVG draw.io All views
Source Mezzanine ~36 GB/source hour Delivery manifest Landing Immutable landing write-once, locked Technical profile Derived decision Complexity profile Recipe content-addressed Work Chunk tasks 220 k/day Chunk outputs idempotent Rendition Elementary streams Quality scores Protected Encrypted segments immutable, 30 d cache Manifests mutable, 30 s cache Consumers CDN + players Cost & quality analytics BigQuery verify then accept digest per chunk gate decisions playback Data Flow — One Master Through the Pipeline Data store Queue / topic External / third party synchronous batch event / async Only the first three columns are irreplaceable. Everything from chunk outputs rightwards is derivable from them. v 1.0 · owner Media Platform Architecture

The line that matters

  • Everything from chunk outputs rightwards is derivable from the immutable landing plus the recipe. That is not an observation about this diagram; it is the design (ADR-01).
  • It is also why chunk scratch carries a 48-hour TTL and the rendition bucket carries lifecycle rules, while the master bucket carries a seven-year retention lock.

Two caches with opposite rules

  • Segments are immutable and cached 30 days. Manifests are mutable and cached 30 seconds.
  • That asymmetry is what makes a manifest revision — adding rungs, narrowing a ladder, withdrawing a title — cost nothing at the edge.

Assumption

  • Assumed mezzanine at 80 Mbps, about 36 GB per source hour. A higher-bitrate delivery standard changes the ingest bill and nothing else in this flow.
◀  Integration Surface All views Storage Zones  ▶
Solution Architecture v1.0 · Google Cloud · Media Platform Architecture · 2026-10 · 22 views · 15 architecture decision records10-encode-data-flow