Streaming Video Encoding & Packaging Pipeline  ·  View 13 of 22  ·  Runtime

Critical Flow — Master to Manifest

Seventeen messages, two of which are failures, and both of those are normal.

Editable source SVG draw.io All views
Content ops Job API Recipe plane Lane scheduler Encode worker QC gate Packager Publisher 1. submit (source, lane, deadline) 2. analyse + solve ladder 3. recipe digest 4. admit job, check spend ceiling 5. order by deadline and work left 6. chunk task + build digest 7. refuse on build mismatch 8. reclaimed: one chunk lost 9. reschedule same chunk 10. stitched rendition 11. score, structure, decoder matrix 12. below floor: new recipe version 13. gate passed 14. segment, encrypt, build manifest 15. manifest candidate 16. verify every segment durable 17. published, atomically Critical Flow — Accepted Master to Published Manifest Two of the seventeen messages are failures, and both are normal: a reclaim and a gate rejection. v 1.0 · owner Media Platform Architecture

The two failures are the design

  • A reclaim costs one chunk and is rescheduled without operator action. It is drawn in the main flow because it happens continuously, not exceptionally.
  • A gate rejection returns to the recipe plane, not to the scheduler. The retry is a new recipe version — re-running the same one is deterministic and would fail identically.

Two checks that refuse rather than warn

  • A worker whose encoder build does not match the recipe refuses the task. Producing an output that cannot be reproduced is worse than producing none.
  • The publisher verifies every referenced segment is durably present before emitting a manifest. A manifest pointing at an absent segment is a publication defect, not a cache miss.

Spend is checked at admission

  • The ceiling is enforced when the job is admitted, as a hard stop. An alert on a runaway retry loop arrives after the money is spent.