Streaming Video Encoding & Packaging Pipeline  ·  View 18 of 22  ·  Operations

Observability

Six signal types across six stages, with cost as a signal row rather than a monthly report.

Editable source SVG draw.io All views
Intake Recipe Encode Gate Package Publish & serve Metrics Accept / reject rate Analysis cost / hour Chunk-seconds / s Reclaim rate Pass rate by rung Key fetch latency Manifest p99 JIT first-byte p95 Logs & audit Conformance verdicts Override actor + reason Build-mismatch refusals Waivers Key id, never key Swaps and rollbacks Traces Upload to accept Analyse to digest Task claim to output Score fan-in Segment to manifest Request to first byte SLOs Verdict within 1 h p95 ≤ 0.80× duration False pass ≤ 0.2% Control plane 99.95% Rollback ≤ 120 s Cost signals TB landed / day Compute vs egress saved Cost per source hour Re-encode cost of failures Cost per delivered hour JIT share of spend Alarms Supplier silent 24 h Progress stalled Spend ceiling hit Quarantine opened Key service degraded Nearest-rung fallback rate Observability — Signal by Pipeline Stage Cost is a signal row, not a monthly report: at 40 PB/day of egress the ladder decision is an operational control. v 1.0 · owner Media Platform Architecture

Cost is an operational signal here

  • At 40 PB/day of egress, the ladder decision has an hourly price. Cost per source hour and cost per delivered hour sit beside latency and pass rate because they are controls, not accounting.
  • The JIT share of spend is the single number that says whether the eviction policy is tuned correctly.

Two alarms that are unusual

  • "Supplier silent 24 h" — an absence of deliveries is indistinguishable from health to a monitor that only watches for failures.
  • "Nearest-rung fallback rate" — the fallback is a correct behaviour, so it will never page anyone unless it is explicitly measured.

What is audited rather than logged

  • Recipe overrides with actor and reason, gate waivers, swaps, rollbacks and takedowns go to the append-only audit log. Key ids are recorded; keys never appear in any signal.