File Upload & Scanning Pipeline  ·  View 02 of 21  ·  Context and scope

High-Level Architecture

Five stages, and the seam between Decide and Serve that the rest of the set exists to explain.

Editable source SVG draw.io All views
Admit Upload API initiate · finalise Credential minting user-delegation SAS, 1 path Transfer Untrusted plane Blob, no read path Session & chunk state resumable 7 days Identify Finalise & verify chunk set + whole-object hash Content identity hash, type by inspection Decide Priority scan queues interactive · background · bulk Sandboxed scan workers ephemeral, egress-restricted Verdict store hash × engine version Serve Lifecycle state machine the only gatekeeper Serving plane available objects only Edge delivery Front Door enqueue once verdict row promote read SAS High-Level Architecture — Five Stages, One Seam Interface / broker Security / platform Data store Application we own Queue / topic event / async synchronous Bytes (dashed) go client-to-store and never through platform compute; only control passes through the Upload API. The seam is between Decide and Serve: bytes exist from stage 2, access begins at stage 5. v 1.0 · seam state, not storage, grants access

Decisions

  • Bytes go client-to-store on a path-scoped credential; only control passes through platform compute, on either direction of travel
  • Finalisation is an explicit assertion of the chunk set and the whole-object hash — the platform refuses to assemble a mismatch rather than storing a truncated object
  • An object is enqueued for scanning exactly once, durably, and cannot reach a terminal state without a verdict recorded against it

The seam

  • Bytes exist from stage 2. Access begins at stage 5. Everything between is the platform holding an object it will not let anyone read.
  • That gap is why ingest availability (99.99%) and scan availability (99.95%) are separate numbers, and why a scanner outage is a latency incident rather than an availability one
  • It is also the cost: a 50 GB object is durable and useless for up to twenty minutes, and the product has to say so honestly

Targets (stated assumptions)

  • Initiate p99 ≤ 120 ms; chunk acknowledgement p99 ≤ 400 ms at 16 MB
  • Verdict p50 ≤ 2 s and p95 ≤ 8 s under 10 MB on the interactive lane
  • 700 initiations/second steady, 3,000/second for a ten-minute burst, 45 GB/s aggregate ingress
◀  System Context All views Actors and Their Core Journeys  ▶
Solution Architecture v1.0 · Microsoft Azure · Integration Platform Architecture · 2026-1002-high-level