Audit Log Service  ·  View 02 of 40  ·  1 · Context and scope

High-Level Architecture

Seven stages from a producer's call to a proof an outsider can check — and the one feedback edge that surprises people.

Editable source SVG draw.io All views
Produce Producer SDK spool ≥ 1 h Workload Identity Entra · SPIFFE Accept Ingest API Rust · p99 150 ms Validate + Detect schema · secrets Dedup Window Valkey · 24 h Buffer Apache Kafka 72 h · RF 3 Seal Sealer Parquet · ≤ 60 s Chain Service Merkle per tenant Checkpoint Signer HSM · 15 min Custody Immutable Segments Blob WORM · locked Checkpoint Ledger separate custody Find Indexer commit → 60 s ClickHouse Index derived · rebuildable Query API every read audited Prove Verifier intact · broken here Export Service signed bundles Offline Verifier published binary read is a record rebuild source Audit Log Service — High-Level Architecture External / third party Security / platform Application we own Data store Queue / topic event / async batch The acknowledgement is given at the buffer, not at the segment. What that promise means, and what it does not, is the subject of view 20. v 1.0 · owner Data & AI Global Practice · date 2026-09

Decisions

  • The acknowledgement is issued at the durable buffer, not at the sealed object. That choice buys the 150 ms p99 and costs a bounded window in which the record is durable but not yet locked — view 20 states exactly what the promise covers.
  • Sealing and indexing are independent consumers of the same buffer, so a stalled index never stalls custody and a stalled sealer never stalls the query path.
  • Reading is a write. Every query re-enters through the ingest plane as a new record, which is why the feedback arrow points backwards to the front of the diagram.

Targets

  • Ingest acknowledgement p50 ≤ 25 ms, p99 ≤ 150 ms; ingest plane availability ≥ 99.99%.
  • Commit to queryable p95 ≤ 60 s, p99 ≤ 5 min. Sealing ≤ 60 s, checkpoints every 15 min.
  • Point query over 30 days p95 ≤ 2 s; aggregate over the online window p95 ≤ 30 s; chain verification of 24 h ≤ 90 s.

Omitted here

  • Failure paths — act 7 carries all eleven named classes.
  • Retention, hold and disposal appear as governance in act 5; this view is the happy path from write to proof.
◀  System Context All views Four Planes and the Rule Between Them  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 40 views · Microsoft Azure and open source02-high-level-architecture