# File Upload & Scanning Pipeline

**Solution Architecture v1.0 · Microsoft Azure · Integration Platform Architecture · 2026-10 · 21 views · 13 architecture decision records**

Everyone has used this system without being told its name. The paperclip in the email client. The drag-and-drop into a chat channel that posts a thumbnail a second later. The folder that syncs from a laptop on hotel wifi and is somehow complete in the morning. The CV uploaded to a company that has never heard of you. The 40 GB video edit dropped into a shared drive the night before a deadline. The screens are easy. What is hard is that each of them is a stranger's bytes entering a system that will later hand them to someone who trusts the system more than they trust the stranger — and the obvious design, scan it inline and store it if it is clean, fails three ways at once: it cannot survive hours of bad network, it cannot survive a scanner that is permanently slower than the uploaders, and it has no answer at all when a signature published tomorrow matches a file declared clean today. This package is the internal upload plane for an assumed collaboration SaaS: 40 million monthly active members across 12 product tenants, 60 million objects a day at a mean of 2.4 MB and a maximum of 50 GB, 150 TB/day of ingress, 12 PB under management — on two Azure Blob storage accounts split into an untrusted and a serving plane, user-delegation SAS as the only upload and download credential, Event Grid and three Service Bus lanes for scan fan-out, ephemeral Container Apps jobs in their own subscription as sandboxed scan workers, Cosmos DB for object state and verdicts, and immutable blob storage for the transition log.

The product is not the ability to store a file. It is **the ability to hold an object nobody may read, decide about it, and change that decision later** — which means the honest deliverable is not a guarantee that nothing malicious is ever stored, but a guarantee that nothing malicious is ever *reachable*, with a bounded window when yesterday's judgement turns out to be wrong.

The design rests on one rule: **the object's lifecycle state, not the presence of its bytes in storage, is the sole authority on access — and the untrusted and serving planes are separate, with nothing crossing without a current verdict.**

The decisions that carry the design:

- **State grants access; storage does not.** One state machine is the only thing a read path may consult, so "is this object reachable" has one answer in one place rather than a flag every new caller must remember to check (ADR-01).
- **Untrusted and serving are separate storage accounts.** The isolation claim is structural and auditable — and promotion is therefore a real, priced copy at p99 object size, which is a cost the design names rather than hides (ADR-02).
- **Bytes go around the platform; control goes through it.** Clients write directly to the store on a write-only, single-path, 60-minute credential, so no compute is sized to 45 GB/s and resumability is inherited from the store (ADR-03).
- **Ingest availability is independent of scan availability.** An object is acceptable and durable while the scan tier is wholly down; it simply stays unreachable. A scanner outage is a latency incident, not an availability one (ADR-04).
- **A verdict is a versioned, timestamped, revocable claim.** Not a boolean on the object — because a boolean cannot say which signature version cleared a file, and cannot be corrected without erasing the evidence that the earlier judgement was made (ADR-05).
- **A breached bound yields indeterminate, never clean.** Time, memory, archive depth and expansion ratio are all bounded, and a decompression bomb is a policy outcome naming the bound it hit rather than a crashed worker (ADR-06).
- **The scan worker is the least-trusted compute in the platform.** Its input is chosen by the adversary, so it runs in its own subscription, in ephemeral per-object jobs, with no inbound reachability and no credential able to change an object's state (ADR-07).
- **Back-pressure is declared in advance, lane by lane.** Bulk refuses first, background throttles, interactive is rejected at initiation — and the one promise never traded is that nothing reaches a reader without a verdict (ADR-08).
- **Verdict reuse is bounded by a declared isolation level.** Deduplication is the biggest cost lever here and also turns the index into a cross-tenant existence oracle, so the default is within-tenant and platform-wide reuse is a knowing opt-in (ADR-09).
- **Finalisation is an assertion the platform may refuse.** The client asserts the block set and the whole-object hash; a mismatch is a failed upload, never an assembled object, which closes partial-commit scan evasion (ADR-10).
- **Every object reaches a terminal state or a report.** Every hop in this pipeline can lose an object without anything failing, so a reconciler sweeping dwell-time breaches is a first-class component rather than a cron job (ADR-11).
- **Residency is enforced by the absence of any replication relationship.** There is no configuration that could be changed to make a cross-boundary copy happen, because there is no relationship to misconfigure (ADR-12).
- **Object size is a workload class, not a parameter.** A 50 KB screenshot and a 50 GB archive share an API and nothing else, so there are three execution profiles and three published time-to-verdict targets (ADR-13).

The architecture one-pager (including why the design should still hold up in ten years, what a four-week prototype should prove, and the six risks that would change it) and the full decision record appear on the landing page of the diagram set, directly below the index of views. The same content is published as [docs/architecture-one-pager.md](docs/architecture-one-pager.md) (~13 min) and [docs/decision-record.md](docs/decision-record.md) (~49 min).

---

## What is here

| Path | Contents |
|---|---|
| `diagrams/index.html` | The landing page: 21 views in seven acts with every format linked, then the **architecture one-pager** and the **decision record** |
| `diagrams/*.html` | One self-contained page per view: the inlined diagram plus the reasoning cards, with copy / PNG / PDF export |
| `diagrams/svg/*.svg` | The same views as SVG with the diagram XML embedded; they re-open fully editable in diagrams.net |
| `diagrams/drawio/*.drawio` | draw.io source |
| `docs/architecture-one-pager.md` | The one-pager as markdown |
| `docs/decision-record.md` | The 13 decision records, the capability-to-technology table and the package glossary as markdown |
| `specs/part-a..c.json` | Diagram specifications, the source of truth for every view |
| `specs/manifest-a..c.json` | Acts, page titles, subtitles and reasoning cards |
| `specs/adr-onepager.json`, `specs/adr-records-a..b.json` | The one-pager, the decision records, the capability-to-technology table and the glossary |
| `scripts/build.sh` | Rebuilds every deliverable from the specs (Node only, no network) |
| `ask.md` | The requirement |

## The twenty-one views

| # | View | What it answers |
|---|---|---|
| 01 | System Context | Fifteen things outside the boundary, and why the public form submitter is drawn beside the members rather than beside the product backends |
| 02 | High-Level Architecture | Five stages, and the seam between Decide and Serve that the rest of the set exists to explain |
| 03 | Actors and Their Journeys | Nine actors, three of them machines, and the one whose experience is the design centre |
| 04 | Journey — Attach a File | Four megabytes, five phases, and two seconds that must not feel like doubt |
| 05 | Journey — A 40 GB Master | The case that kills naive designs: an interruption that is certain and a restart that is unacceptable |
| 06 | Journey — Review a Block | Whether quarantine is a security control or a help-desk queue, decided in one journey |
| 07 | Layered Architecture | Eight layers, two of them storage, and the only arrow in the set that runs backwards |
| 08 | Platform Components | Three subscriptions, because the scan plane's blast radius is a subscription boundary and not a network rule |
| 09 | Interface Catalogue | Four contracts in, three out, and the one that carries bytes without touching the platform |
| 10 | Data Flow | One file to the first read, the 35% that never reaches a scanner, and saturation expressed as refusal |
| 11 | Storage Classes | Three classes ordered by what loss costs; only one has an RPO worth arguing about |
| 12 | Data Model | Twelve entities, and the thirteenth — `is_clean` — that every first draft adds and this design refuses |
| 13 | Critical Flow | Twenty-one messages, and the two most designs leave out: the honest 202, and the re-check at download |
| 14 | Scan Pipeline by Class | Five rows, six stages, and the blank cells that are the argument |
| 15 | Back-Pressure | Four load bands, three lanes, and the one invariant that is never traded |
| 16 | Download Path | Four checks before any byte moves, and the five minutes that bound a revocation |
| 17 | Deployment | Two regions inside one residency boundary, and the arrow that is deliberately absent |
| 18 | Observability | Six signal families across six stages, reduced to four alarms — and why queue depth is not one |
| 19 | Object Lifecycle | Seven states, and the closing arrow most implementations never draw |
| 20 | Security Zones | Six zones, and the two crossings that carry the entire security argument |
| 21 | Identity & Credential Flow | Nineteen messages establishing that no credential outlives the decision that justified it |

## Rebuilding

Node 20+ and nothing else. No draw.io Desktop, no browser, no network.

```bash
bash scripts/build.sh
```

The build generates draw.io from the specs, fails on a geometry or routing defect, renders editable and plain SVG, writes the HTML pages and the index, injects the one-pager and the decision record into the landing page, regenerates `docs/*.md`, and proves every relative link resolves. The current build passes both gates with **0 errors and 0 warnings**, 160 edges routed, and 368 of 368 nodes resolving to an icon.

## Limits of this package

- It is a **design**, not a report on a running system. Every rate, latency, volume, retention and cost figure is a stated assumption, sized for the reference workload described above.
- Three of the ask's eight Core Architecture Questions are shown resolved one way in the views while remaining genuinely open in the requirement: large-object scanning is drawn as assembled rather than streamed, the engine mix is drawn as fast-always with deep-conditional, and promotion is drawn as a copy between accounts. Each is flagged on the view that assumes it, and each has a named flip condition in its decision record.
- The quarantine review console, the appeal workflow and the cost-attribution surface are specified but not designed to screen level — they are product surfaces this package treats as consumers of its state.