Streaming Video Encoding & Packaging Pipeline  ·  View 22 of 22  ·  Assurance

Failure Classes

Ten assumed failures, how each is detected, and what each one still leaves behind.

Editable source SVG draw.io All views
Assumed failure Detected by Immediate handling Residual exposure Capacity reclaim 60% of fleet in 10 min Lease expiry Requeue one chunk None — expected state Poison source Decoder hangs or crashes Chunk timeout + memory cap Quarantine with diagnosis Title misses its deadline Encoder build drift Worker ≠ recipe build Admission check Worker refuses task Throughput dip on rollout Stitch defect Visible pumping at a join Boundary verification Re-encode chunk pair Subtle artefacts below threshold Gate false pass Scores well, plays badly Playback telemetry Quarantine + manifest revision ≤0.2% of titles per quarter Key service outage KMS or DRM unavailable Key fetch latency alarm Stall publish, retry Premiere slips; never unencrypted Partial publish Some renditions absent Durability pre-check Publish narrower manifest Lower top rung on day one Cold-title demand spike Evicted rungs, sudden load Miss rate by title Coalesce + promote tier First viewers get a lower rung Region loss Encode region unavailable Control-plane health Fail over, regenerate Degraded rung set until refilled Cost runaway Retry loop or bad recipe Spend ceiling at admission Hard stop, not an alert Blocked job needs a human Failure Classes and Their Handling The residual column is the honest one: every row leaves something, and in eight of ten rows what it leaves is quality or lateness, never protection. v 1.0 · owner Media Platform Architecture

The residual column is the honest one

  • Every row leaves something. In eight of the ten, what it leaves is reduced quality or a missed deadline.
  • In none of them is what it leaves reduced protection — that is the invariant the table exists to demonstrate.
  • Two rows end at a human: a quarantined poison source and a blocked job that hit its spend ceiling. Both are deliberate: an automated answer to either would be guessing.

"Flake" is not in this table

  • Capacity reclaim is the only expected-and-benign row, and it is handled structurally rather than by retry policy.
  • Everything else is root-caused: a build mismatch refuses, a poison source quarantines, a stitch defect re-encodes a chunk pair, a false pass quarantines a published rendition.

Assumptions

  • Assumed gate false-pass rate ≤ 0.2% of published titles per quarter, measured by post-publication quarantine — the number the second gate is judged on.
  • Assumed 18% hourly reclaim and 60% fleet withdrawal inside 10 minutes as the steady-state design point, not a worst case.
◀  Identity, Keys and Licence Flow All views
Solution Architecture v1.0 · Google Cloud · Media Platform Architecture · 2026-10 · 22 views · 15 architecture decision records22-failure-classes