File Upload & Scanning Pipeline  ·  View 18 of 21  ·  Operations

Observability

Six signal families across six stages, reduced to four alarms that wake a human.

Editable source SVG draw.io All views
Admit Transfer Finalise Scan Promote Serve Latency initiate p99 chunk ack p99 finalise p99 time-to-verdict by size band promote duration credential p99 Saturation admission rejects / lane in-flight sessions finalise backlog ALARM: oldest unscanned / lane promotion queue edge origin fetches Correctness claimed vs determined type chunk checksum failures hash mismatch refusals indeterminate rate, bound hit ALARM: stuck past dwell ALARM: serve on non-available Security authz denials SAS misuse attempts type-policy rejects detections by engine releases + overrides break-glass reads Freshness ALARM: signature set age re-scan sweep progress revocation propagation Cost per-tenant initiations ingress GB / tenant dedup hit rate scan cost / 1,000 objects promotion copy bytes egress GB / tenant Observability — Six Signals Across Six Stages, Four Alarms Four alarms wake a human. Everything else is a dashboard. Note what is not an alarm: queue depth. A deep queue that is draining is fine; a shallow queue whose oldest item is an hour old is not. v 1.0 · alarms 4

What is not an alarm

  • Queue depth. A deep queue that is draining is fine; a shallow queue whose oldest item is an hour old is not.
  • The scan tier is scaled and alarmed on oldest-unscanned age per lane, because the promise being kept is time-to-verdict rather than throughput
  • CPU and worker count are capacity inputs, not symptoms — alarming on them produces pages nobody can act on

The four alarms

  • Oldest-unscanned age per lane past its band — the saturation signal that drives the shedding order
  • An object in a non-terminal state past its expected dwell time — the stuck-object signal, which is both a support ticket and a security hole
  • A serve attempt against a non-available object, which should be impossible and therefore matters; and signature-set age, which bounds how stale the platform's knowledge is

Measured, not alarmed

  • Claimed-versus-determined content type disagreement rate, which is an early signal of either a buggy client or a deliberate one
  • Dedup hit rate and scan cost per 1,000 objects, because the cost model is a design parameter rather than a monthly surprise
  • Releases, overrides and break-glass reads — low-volume, high-consequence, reviewed rather than paged