Audit Log Service

Architecture Views

40 views, in reading order. Every view ships three ways: an HTML page, an SVG that re-opens in diagrams.net fully editable, and draw.io source.

A shared audit log for the whole estate: it accepts security- and compliance-relevant events from every application and administrative surface, holds them so that nobody — including its own operators and the subscription's administrators — can alter or remove one inside its retention period, and answers investigative and regulatory questions against thirteen months of that history. Read the set in order. Act 1 fixes the boundary and the plane separation everything else follows from; act 2 says who the record is actually for and where each of them is currently let down; acts 3 and 4 give the structure and say which store is the truth and which is merely convenient; act 5 is what happens when the thing is used; act 6 is running and paying for it; act 7 is why it is safe, with the operators inside the threat model rather than outside it. One decision governs the whole set: the write path holds no authority over a record's lifetime, and no principal exists anywhere that can both write a record and remove it.

1 · Context and scope

What sits inside the boundary, who feeds it, who checks it, and the four-plane separation that the rest of the architecture is a consequence of.
03 Capture plane — may create a record, may never remove one Boundary Ingest API write only Validator schema + secrets Hand-off Durable Buffer no delete verb Quarantine rejections, audited Custody plane — may retain and expire, may never create a record Seal and chain Sealer append only Chain Service hash forward Segment Store locked retention Expire Retention Engine extend, never shorten Legal Hold Registry legal role only Disposal Service certificate or nothing Query plane — derived, authoritative for nothing Projection Indexer rebuildable Search Index disposable Access Query API tenant scoped Archive Query async jobs Evidence plane — may read and prove, may never write a record Proof Verifier recompute chain Proof Service inclusion, absence Delivery Export Service watermark, cap Offline Verifier no service needed Managed HSM key custody team Confidential Ledger checkpoint custody Microsoft Entra ID identity team sealed hand-off projection signed head sign, never export Four Planes — No Principal Can Both Write and Remove Application we own Queue / topic Data store External / third party Security / platform event / async synchronous Four planes, four managed identities, four Entra groups, no membership overlap. The capture plane holds no delete verb in its data-access layer; the custody plane holds no insert verb for records. v 1.0 · owner Data & AI Global Practice · date 2026-09 Four Planes and the Rule Between Them The critical design decision drawn as a boundary: which plane may create a record, which may retain and expire one, and which may only prove things about them. HTML page SVG draw.io

2 · People and journeys

Who the record is for, what each of them is trying to get done, and the three journeys that decide whether this service is worth having.
04 The people the record is for Investigator 38 named, scoped Goal — Show me every privileged action this account took last March, and tell me honestly whether I am seeing all of them. Core journeys Trace an account view 05 Scope a breach window Compliance Officer 12 named Goal — Give me something I can hand to a regulator that they can check without ever asking us a question. Core journeys Produce an evidence bundle Certify a disposal External Auditor annual and ad hoc Goal — Prove to me that the record could not have been altered, without me having to trust you or your operators. Core journeys Verify a bundle offline view 06 Challenge a gap in a range The people who run it Platform Operator SRE, on call 24×7 Goal — Hold ingest above 99.99% without ever being able to read a single customer's audit record. Core journeys Absorb a 4× burst Rebuild the index Security Engineer detection and response Goal — Be told within fifteen minutes if anyone has touched a segment, and that includes me. Core journeys Answer a divergence alert Review export anomalies Legal Hold Officer segregated, 4 named Goal — Freeze everything touching this matter today, and be certain nothing expired overnight while I slept. Core journeys Place and lift a hold Prove nothing was lost The systems that write to it Business Application 410 services, one SDK Goal — Emit my audit events without your outage becoming my outage. Core journeys Onboard an event class view 07 Survive an audit outage Admin Console privileged surfaces Goal — Record the privileged action I just performed, including the one that was refused. Core journeys Record a denied action Azure Control Plane Activity Log export Goal — Land every subscription-level change in the same record as the applications, not in a portal of its own. Core journeys Export control-plane events The Service Itself reads, exports, holds Goal — Audit my own use into myself, so nobody reads this log without leaving a record of having read it. Core journeys Write a read record Write a key-use record The machines that act without being asked Rolling Verifier full corpus / 7 days Goal — Walk the whole chain every week and name the first divergence before anyone else finds it. Core journeys Complete a sweep Retention Scheduler nightly window Goal — Dispose of exactly what expired, nothing that is held, and leave a signed certificate behind. Core journeys Run a disposal window Index Reconciler continuous Goal — Find the record that exists but cannot be found, before an auditor does. Core journeys Repair a divergence Cross-Region Replicator continuous, one way Goal — Keep the second region inside RPO 0 while holding no right to delete anything in the first. Core journeys Replicate segments Who the Record Is For, and What Each of Them Gets to Do Person or role Journey / task External / third party Security / platform Four groups, and the fourth is the one usually left off. A weekly verifier and a nightly disposer have SLAs, failure paths and blast radius of their own; leaving them off the cast list is how they end up with no owner. v 1.0 · owner Data & AI Global Practice · date 2026-09 Actors and Their Core Journeys Fourteen actors in four groups, each with the goal they actually arrive with — including the four that are machines. HTML page SVG draw.io
05 Investigator Security incident response Goal — Establish who granted this role, under whose authority, and whether the record is complete Trigger — An alert: a service principal acquired Owner on a production subscription overnight Done when — A defensible timeline handed to the incident lead, with a stated completeness bound 1 · Suspect the alert lands 2 · Scope actor and window 3 · Search ◆ moment of truth 4 · Corroborate ◆ moment of truth 5 · Hand over to incident lead What they do Reads the alert Fixes actor + window Cites justification Queries by actor Widens to resource Checks denied too Pulls before/after Exports the range Where they do it Alert console Query console Query API Query API Export console Behind the glass Alertmanager RBAC scope check ClickHouse index Segment fetch Export service How it feels Confident Working Doubting Where it hurts Scope check is slow Is the index complete? Payload not in index What answers it Precomputed scope Completeness bound Fetched on demand Signed bundle Investigator — Trace an Account Through Thirteen Months The trough is at Search, and it is not about speed. The investigator cannot tell whether an empty result means nothing happened or the index has not caught up — which is why every result carries the sealed segment it is complete to. View 22. v 1.0 · owner Data & AI Global Practice · date 2026-09 Journey — Trace an Account Through Thirteen Months The investigator's path from an alert to a defensible timeline, and the moment where the tool stops being trustworthy to them. HTML page SVG draw.io
06 External Auditor Appointed by the regulator Goal — Satisfy myself that the record could not have been altered, on evidence I can check myself Trigger — The annual controls audit, plus one range subpoenaed under litigation Done when — A signed opinion that the range is intact — or a named, dated divergence 1 · Request names a range 2 · Receive the bundle 3 · Verify offline ◆ moment of truth 4 · Challenge ◆ moment of truth 5 · Conclude opinion issued What they do Names tenant + range Takes the bundle Runs the verifier Checks the anchor Asks about a gap Signs the opinion Where they do it Evidence request Signed bundle audit-verify CLI Disposal certificate Opinion letter Behind the glass Export service Proof service Published key Tombstone record How it feels Satisfied Working Sceptical Where it hurts Trusts nothing said Whose key is this? A gap reads as tamper What answers it Bundle self-contained Third-party anchor Lawfully disposed No service call made External Auditor — Verify Without Trusting the Operator The only phase that rises is the one performed offline. Everything the service says about itself is discounted; what carries the audit is a bundle that verifies with a published algorithm and a public key. Views 24 and 30. v 1.0 · owner Data & AI Global Practice · date 2026-09 Journey — Verify Without Trusting the Operator The only journey in the set whose best moment happens offline, on a machine this organisation does not run. HTML page SVG draw.io

3 · Structure

The layering rule, the deployable units inside each plane, every interface in and out, and the shape of the hash chain that makes tampering provable.
09 Capture plane — namespace audit-ingest · managed identity ingest-mi envoy-gateway Envoy · mTLS · 12 pods ingest-api Rust axum · 24 pods admission schema · secrets · size quota-governor OPA · per tenant quarantine-writer rejections, audited Durable buffer — Strimzi on dedicated node pool kafka-broker 12 × D16s v5 · RF 3 kraft-controller 3 pods, 3 zones tiered-storage 6 h local, 72 h Blob Custody plane — namespace audit-custody · managed identity custody-mi sealer Go · 32 pods chain-service Go · 6, leader per shard checkpoint-signer Go · 3 pods compactor chain-preserving retention-engine Go · 3 pods disposal-service Go · 2 pods Query plane — namespace audit-query · managed identity query-mi indexer Go · 24 pods clickhouse 16 × E16ds v5 query-api Go · 12 pods reconciler Go · 4 pods trino archive, autoscaled Evidence plane — namespace audit-evidence · managed identity evidence-mi verifier Go · 16 pods proof-service Go · 6 pods export-service Go · 6 pods Azure Blob Storage immutable containers Confidential Ledger separate subscription Managed HSM per-tenant keys PostgreSQL Flexible registry, holds Valkey Cluster dedup, quota acks=all, isr 2 consume PUT once append head consume Container Architecture — Deployable Units (C4 L2) Interface / broker Application we own Queue / topic Security / platform Data store synchronous event / async Five namespaces, five managed identities, five Entra groups with no shared membership. Blob data-plane roles are granted per plane: the sealer may create a blob and no more; nothing anywhere holds Storage Blob Data Owner. v 1.0 · owner Data & AI Global Practice · date 2026-09 Container Architecture (C4 L2) Every deployable unit, its technology and its scale, grouped by the plane whose identity it runs under. HTML page SVG draw.io
10 Arrival Network Private Endpoint in-estate producers Front Door + WAF off-estate only Identity mTLS Termination Envoy Token Validation Entra · SPIFFE Tenant Assignment from identity only Admission — every check before the record can be accepted Shape Schema Validation versioned, Apicurio Vocabulary Check controlled verbs Size Cap 256 KB, reject Content Secret Detector Presidio + rules PAN / Credential reject, never redact Governance Idempotency Check 24 h window Tenant Quota token bucket Class Declaration open or closed Commit — the acknowledgement is issued here and nowhere earlier Durable write Kafka Producer acks=all, isr ≥ 2 Partition Router tenant · resource Promise Commit Receipt event_id + offset received_at Stamp service clock Rejection — a refusal is itself an auditable event Structured Error field and reason Quarantine Topic never blocks a partition Quarantine Segments same retention lock oversized secret found durable Ingest Plane — Everything That Happens Before the Acknowledgement Interface / broker Security / platform Application we own Decision point Data store Queue / topic failure / alternate event / async Nothing here transforms the record. Validation and detection can only accept or reject; there is no redaction path, because a record that had to be edited to be storable was never a faithful account of the action. v 1.0 · owner Data & AI Global Practice · date 2026-09 Ingest Plane Every check that happens before an event can be accepted — and the absence of any path that changes one. HTML page SVG draw.io
11 Assemble — a segment is closed at most 60 seconds after it opens Read Partition Reader ordered, committed Segment Grouping tenant · date · class Write Parquet Writer zstd · per-column Seal Trigger 60 s or 256 MB Chain — hashing forward so that removing anything invalidates everything after Hash Leaf Hashing SHA-256, canonical Segment Merkle Root RFC 6962 style Predecessor Link prev_root in header Sign Checkpoint Builder 15 min, all tenants HSM Signature non-exportable TSA Countersign RFC 3161 Lock — immutability is a storage property, not an application rule Object Immutable Container locked time-based Encryption Scope per-tenant CMK Version-Level Lock no overwrite Record of it Segment Registry PostgreSQL, derived Lifecycle Tiering hot → cool → archive Expire — acting only on records this plane did not create Retention Class 13 mo · 7 yr · 10 yr Hold Check tenant, actor, resource Crypto-Shred destroy segment DEK Disposal Certificate signed, re-ingested Tombstone Node chain stays verifiable Confidential Ledger chain heads only signed head Custody Plane — Seal, Chain, Lock, Expire Application we own Decision point Security / platform External / third party Data store synchronous The lock is set by a locked time-based retention policy on the container. Once locked, the policy can be extended and cannot be shortened, deleted or bypassed — not by the subscription owner, not by a support ticket, and not by the plane that wrote the object. v 1.0 · owner Data & AI Global Practice · date 2026-09 Custody Plane Assemble, chain, lock, expire — the four things that happen to a record after nobody can change it any more. HTML page SVG draw.io
13 Level 1 — the record Canonical Bytes stored form, never re-encoded Leaf Hash SHA-256 with domain tag Leaf Index position within segment Level 2 — the segment, sealed at most every 60 seconds Merkle Tree RFC 6962 hashing Segment Root one per sealed object Segment Header prev_root, count, range Object Digest binds bytes to root Level 3 — one chain per tenant, so a proof reveals no other tenant Tenant Head hash of prev head + root Parallel Sealing 500 chains, no contention Tenant-Blind Proof path stays inside tenant Tombstone Link survives disposal Level 4 — one global checkpoint every 15 minutes binds every tenant Checkpoint Tree over all tenant heads HSM Signature non-exportable key TSA Countersign external clock Ledger Publication separate custody Public Witness Log Rekor · phase 3 leaves extends anchors phase 3 anchor The Shape of the Chain — Per-Tenant, Anchored Globally Application we own Data store Security / platform External / third party synchronous event / async Question 1 answered: per-tenant chains anchored into a global checkpoint. One global chain would serialise sealing at 100 k events/s and leak cross-tenant volume through every Merkle path; independent per-tenant chains would leave a tenant-scoped attacker a small structure to forge. v 1.0 · owner Data & AI Global Practice · date 2026-09 The Shape of the Hash Chain Four levels from a record's bytes to a signed global checkpoint — the answer to the first core question in the requirement. HTML page SVG draw.io

4 · Data

Which tier is the system of record and which is disposable, the canonical record, the model behind a proof, and where the tenant boundary sits in storage.
15 tenant tenant_id PK residency_region retention_policy_id FK key_uri (HSM) index_window_days retention_class class_id PK online_months archive_years disposal_grace_days = 30 extendable (never reduced) legal_hold hold_id PK scope_type tenant|actor|resource|range scope_value matter_ref opened_by (legal role only) actor actor_id PK principal_type human|service principal auth_method on_behalf_of (nullable) audit_record event_id PK (service-assigned) tenant_id FK (from identity) occurred_at (producer asserts) received_at (service clock) clock_divergence_s action (controlled verb) outcome success|denied|error idempotency_key schema_version retention_class_id FK fail_mode open|closed resource resource_id PK resource_type stable_identifier tenant_id FK source source_id PK network_origin user_agent session_id request_correlation_id payload payload_ref PK before_value after_value bytes (record cap 256 KB) secret_scan_verdict authority authority_id PK authority_kind policy|role|grant policy_ref grant_ref approval_ref 1 : N 1 : N N : M 1 : N 1 : N 1 : N 1 : 1 1 : N The Audit Record — One Canonical, Versioned Schema tenant_id and received_at are assigned by the service and are the only fields a producer cannot influence. occurred_at is kept exactly as asserted, never corrected — the divergence is recorded beside it instead. v 1.0 · owner Data & AI Global Practice · date 2026-09 The Audit Record One canonical, versioned schema — and the small number of fields the producer is not allowed to influence. HTML page SVG draw.io

5 · Runtime

Ingest and what the acknowledgement promises, sealing and signing, querying and the record a query leaves, verification, export, disposal, legal hold, an audit-plane outage and an index rebuild.

6 · Operations

Where it runs, how it survives a region, how a change to the rules is approved, what is watched, what it costs, and the operator's loop over a system they cannot read.
29 Azure · West Europe Availability zone 1 AKS Ingest Pool 3 × D16s v5 Kafka Brokers 4 of 12 ClickHouse Shards replica set A Custody Pool sealer, chain Availability zone 2 AKS Ingest Pool 3 × D16s v5 Kafka Brokers 4 of 12 ClickHouse Shards replica set B Custody Pool sealer, chain Availability zone 3 AKS Ingest Pool 3 × D16s v5 Kafka Brokers 4 of 12 ClickHouse Shards replica set C Custody Pool sealer, chain Zone-redundant by construction — no single zone holds the only copy Blob Storage ZRS immutable containers PostgreSQL Flexible zone-redundant HA Managed HSM 3-zone pool Front Door + WAF global anycast Private Endpoints in-estate producers Separate subscription, separate administrators, separate break-glass Confidential Ledger chain heads Checkpoint Signing Key own HSM pool Ledger Admin Group no segment access Microsoft Entra ID global tenant Timestamp Authority third party seal signs heads Deployment — One Region, Three Availability Zones Security / platform Queue / topic Data store Interface / broker External / third party event / async synchronous Loss of one zone removes a third of ingest capacity and no data: Kafka keeps isr ≥ 2, ClickHouse keeps two replicas, and the segment store is zone-redundant beneath the immutability policy. Producers see nothing but a brief drop in a connection pool. v 1.0 · owner Data & AI Global Practice · date 2026-09 Deployment — One Region, Three Zones What runs where, what is zone-redundant by construction, and which subscription deliberately is not ours. HTML page SVG draw.io

7 · Assurance

Trust zones, identity, separation of duties, key custody, what each insider attempt actually meets, and every named failure mode with its required handling.
35 Internet — assumed hostile Off-Estate Producer partner systems Auditor Workstation offline verification Published Key + Algorithm deliberately public Perimeter — holds no key Front Door + WAF rate limit, TLS 1.3 Private Endpoint in-estate default path Envoy mTLS SVID or Entra token Capture — write only Ingest API ingest-mi Validator + Detector reject or accept Kafka Buffer no delete API exposed Custody — no standing access Sealer + Chain custody-mi Segment Store create-only role Retention Engine extend only Evidence and query Query API query-mi, tenant scoped Verifier read-only on segments Export Service capped, watermarked Key and checkpoint custody Managed HSM non-exportable Confidential Ledger append only Custody Admin Group disjoint membership TLS 1.3 + workload token principal, tenant, write scope consume, in-cluster mTLS read-only SAS-free RBAC sign · Crypto User only signed bundle, out of band Trust Zones — Every Crossing Authenticated, Every Zone Narrower External / third party Security / platform Interface / broker Application we own Queue / topic Data store synchronous event / async Zones narrow left to right and every crossing is mutually authenticated, in-cluster included. The perimeter can see a record in flight and can read none at rest; the custody zone has no standing human access at all; and the last edge leaves unauthenticated on purpose, because the bundle is built to be verified by someone holding no credential of ours. v 1.0 · owner Data & AI Global Practice · date 2026-09 Trust Zones Six zones of decreasing exposure, every crossing authenticated — and the one edge that deliberately is not. HTML page SVG draw.io
Open svg/<view>.svg or drawio/<view>.drawio in draw.io Desktop or at app.diagrams.net to edit. The SVG carries the diagram inside it, so it is both the picture and the source. This folder is self-contained — copy it whole and every link still resolves.