# Audit Log Service

**Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 40 views · Microsoft Azure and open source**

A shared audit log for the whole estate: it accepts security- and compliance-relevant events from every application, platform service and administrative surface, holds them so that nobody — including its own operators and the subscription's administrators — can alter or remove one inside its retention period, and answers investigative and regulatory questions against thirteen months of that history. The requirement named Amazon Web Services; this set answers it on Azure and open source, substituting Blob immutable containers with a locked time-based retention policy for S3 Object Lock, Apache Kafka on Strimzi for Kinesis, Key Vault Managed HSM for KMS and CloudHSM, Azure Confidential Ledger for the separately administered checkpoint store, ClickHouse for the online index, Trino for the archive path, and Presidio with a compiled rule set for pre-commit secret detection.

One decision carries the whole design: **the write path holds no authority over the record's lifetime.** Capture, custody, query and evidence are four planes with disjoint credentials, and only the custody plane — which cannot create records — can express retention. No principal exists anywhere that can both write a record and remove it. That boundary is why immutability is enforced by a storage retention lock rather than application logic, why the checkpoint ledger sits in a different subscription under different administrators, why the query index is disposable and authoritative for nothing, and why the service's own operators are inside the threat model rather than outside it.

---

## What is here

| Path | Contents |
|---|---|
| `diagrams/index.html` | The view index — 40 views grouped into seven acts, every format linked |
| `diagrams/*.html` | One self-contained page per view: the inlined diagram plus the reasoning it deliberately omits, with copy / PNG / PDF export |
| `diagrams/svg/*.svg` | The same 40 views as SVG with the diagram XML embedded — re-opens in diagrams.net fully editable |
| `diagrams/drawio/*.drawio` | draw.io native source |
| `specs/views.json` | Diagram specifications — the source of truth for every view (assembled from `part-a..g.json`) |
| `specs/manifest.json` | Acts, page titles, subtitles and the decision / assumption / risk cards (assembled from `manifest-a..b.json`) |
| `scripts/build.sh` | Rebuilds every deliverable from the specs |
| `scripts/pin-icons.mjs` | Pins one mark per component so the same box carries the same icon on every page |
| `ask.md` | The original requirement |

688 icons are embedded in the files, so the deliverable renders identically anywhere with no network request. Each view page links to its own SVG and draw.io source, back to the index, and to the previous and next view.

To rebuild after editing a spec:

```bash
node scripts/pin-icons.mjs     # only after adding new component labels
bash scripts/build.sh          # Node 20+ and nothing else
```

Edit `specs/part-a.json` … `part-g.json` and `specs/manifest-a.json` / `manifest-b.json`; the build script assembles `views.json` and `manifest.json` from them and fails if the two lists disagree or a view names a section that does not exist.

---

## The seven acts

| Act | What it lands | Views |
|---|---|---|
| 1 · Context and scope | The boundary, who feeds it, who checks it, and the four-plane separation everything else follows from | 01–03 |
| 2 · People and journeys | Who the record is for, and the three journeys that decide whether the service is worth having | 04–07 |
| 3 · Structure | The layering rule, the deployable units, every interface, and the shape of the hash chain | 08–13 |
| 4 · Data | Which tier is the system of record and which is disposable, the canonical record, and where the tenant boundary sits | 14–19 |
| 5 · Runtime | Ingest, sealing, querying, verification, export, disposal, legal hold, an audit-plane outage and an index rebuild | 20–28 |
| 6 · Operations | Where it runs, how it survives a region, how the rules are changed, what is watched, what it costs | 29–34 |
| 7 · Assurance | Trust zones, identity, separation of duties, key custody, insider attempts and every named failure mode | 35–40 |

---

## The seven core questions, answered

The requirement poses seven questions and refuses to answer them. This set commits to all seven.

| # | Question | The answer here | View |
|---|---|---|---|
| 1 | What is the shape of the hash chain? | Per-tenant chains anchored into a signed global checkpoint every 15 minutes. Not one global chain (serialises sealing, leaks cross-tenant volume through every proof path) and not independent per-tenant chains (too small a structure to forge). | 13 |
| 2 | Does the audited action fail closed when the audit plane is unavailable? | Per event class, declared in the schema registry and unchangeable by the producer at runtime. `role.grant` fails closed; `record.read` fails open and emits a gap marker that is itself an audit record. | 27 |
| 3 | Where does a record become immutable? | Acknowledgement at the durable buffer, which promises zone-loss durability and sealing within 60 s, not a retention lock. A two-stage receipt lets a producer that needs custody-grade certainty wait for the seal. | 20 |
| 4 | One store or two? | Two, with divergence made measurable rather than impossible. Every query result carries the sealed segment it is complete to, so the index is never quietly treated as authoritative. | 28 |
| 5 | How is retention expiry executed, and can deletion be proved? | Both, in order: crypto-shred the segment key at expiry, then delete the ciphertext once the retention lock allows it. A tombstone keeps the Merkle root and the certificate, so a disposed range verifies as *lawfully disposed*, not as tampering. | 25 |
| 6 | What does the log claim about time and order? | Ordering by commit on the service's own clock. `occurred_at` is preserved exactly as asserted, never corrected, with the divergence recorded beside it. An external RFC 3161 countersignature stops the service moving its own clock. | 15, 21 |
| 7 | Where is the tenancy boundary in storage? | A split: 486 tenants share containers with a per-tenant encryption scope and key; 14 with residency or contractual requirements get dedicated accounts. A misrouted read returns bytes the reader holds no key for. | 19 |

---

## Three findings the sponsor should read before build

1. **The cost target is not reachable.** The requirement's $0.09 per million events, all-in for 13 months, implies about $5.9 k a month at 66 billion events a month. The segment store alone is roughly $4.1 k of that before a single core is switched on, and a fixed floor of about $15.6 k — HSM in two regions, the ledger, PostgreSQL, observability, a minimum broker and cluster footprint — does not scale down with volume. The defensible figure for this design is **$0.65 per million** with a 90-day index and $0.75 with a full 13-month index. Every property that could be dropped to approach $0.09 is a property the requirement asks for by name elsewhere. View 33 carries the arithmetic.

2. **Blob locked immutability is the single most load-bearing platform assumption in the set.** It is treated as equivalent in force to S3 Object Lock in compliance mode, and Managed HSM purge protection is treated as making a crypto-shred irreversible. Both should be confirmed with Microsoft in writing, against the erasure standard the applicable regulators actually accept, before the first disposal window runs. Views 11, 25 and 38.

3. **The fail-closed event class list changes 410 other services' SLOs, not only this one.** It must be agreed with those owners rather than declared here. View 27.

---

## The 40 views

| # | View | Layout | What it answers |
|---|---|---|---|
| 01 | System Context | context | Who writes, who reads, who checks, and which custody is deliberately held elsewhere |
| 02 | High-Level Architecture | flow | Seven stages from a producer's call to a proof an outsider can check |
| 03 | Four Planes | nested | The critical design decision drawn as a boundary |
| 04 | Actors and Their Core Journeys | actors | Fourteen actors in four groups, each with the goal they arrive with |
| 05 | Journey — Trace an Account | journey | Where an investigator stops trusting the tool, and what answers it |
| 06 | Journey — Verify Without Trusting Us | journey | The only journey whose best moment happens offline |
| 07 | Journey — Emit Without Our Outage | journey | The phase where an architecture question wears a UX costume |
| 08 | Layered Architecture | bands | What depends on what, and the two sideways arrows |
| 09 | Container Architecture (C4 L2) | nested | Every deployable unit, its technology and its scale |
| 10 | Ingest Plane | nested | Every check before acceptance, and the absence of a redaction path |
| 11 | Custody Plane | nested | Assemble, chain, lock, expire |
| 12 | Integration Surface | hub | Every way in and out, and no interface that is both |
| 13 | The Shape of the Hash Chain | nested | Four levels from a record's bytes to a signed checkpoint |
| 14 | Data Architecture | nested | Four tiers, four durability contracts |
| 15 | The Audit Record | er | One canonical schema, and the fields a producer cannot influence |
| 16 | Segment, Chain and Certificate | er | What makes a range provable, including after disposal |
| 17 | Data Flow — Commit to Answerable | flow | What happens between the acknowledgement and being findable |
| 18 | Retention and Tiering | grid | What each store does at each age, and what never changes |
| 19 | Where the Tenant Boundary Sits | nested | Shared containers, dedicated accounts, and the residual risk |
| 20 | Synchronous Ingest | sequence | What the acknowledgement actually promises |
| 21 | Seal, Chain, Sign, Publish | sequence | One 15-minute checkpoint, and the HSM outage path |
| 22 | An Investigative Query | sequence | The message deliberately ordered before the query runs |
| 23 | Verification | flow | Intact, broken here, or lawfully disposed — never a boolean |
| 24 | Evidence Bundle and Offline Verification | sequence | Twelve messages, the last two involving nothing we run |
| 25 | Disposal | cycle | A governed, evidenced act that a range still verifies through |
| 26 | Legal Hold | flow | Suspending disposal without touching a record |
| 27 | When the Audit Plane Is Unavailable | swimlane | Fail open or closed, declared per event class |
| 28 | Losing the Index | flow | Proving the claim the whole set rests on |
| 29 | Deployment — One Region, Three Zones | nested | What runs where, and which subscription is not ours |
| 30 | Two Regions, Custody Belonging to Neither | nested | Failover, one-way replication, and the anchor outside both |
| 31 | Shipping It, Changing the Rules | flow | The CI gate that enforces the central principle |
| 32 | Observability | grid | Signal type by stage, and the deliberate emptiness of the stack |
| 33 | Cost | flow | Where the money goes, and why the target is seven times short |
| 34 | The Operator's Loop | cycle | Running a system you are forbidden from reading |
| 35 | Trust Zones | zones | Six zones of decreasing exposure, and the one unauthenticated edge |
| 36 | Identity and Credentials | sequence | Why the service records the identity it observed, not the one it was told |
| 37 | Roles and Separation of Duties | grid | Eight roles, seven capabilities, and blanks that are the design |
| 38 | Key Custody | nested | Three kinds of key, and the combinations nobody holds |
| 39 | The Operators Are In Scope | swimlane | Six insider attempts and what each actually meets |
| 40 | Named Failure Modes | nested | Sixteen failures in four categories, sorted by permitted response |

---

## Build status

```
40 views · 0 spec errors
validator (strict)   40 file(s) — 0 error(s), 0 warning(s)
route check          245 edge(s) routed — 0 error(s), 3 warning(s)
icons                649 node(s) — 0 unresolved, 0 weak, 688 embedded
link check           41 page(s), 318 relative links — all resolve
```

The three routing warnings are label-proximity clutter on views 01, 12 and 14, each on a diagram where the alternative was deleting an edge that carries meaning. No arrow crosses a box and no label sits on one anywhere in the set.