Audit Log Service  ·  View 09 of 40  ·  3 · Structure

Container Architecture (C4 L2)

Every deployable unit, its technology and its scale, grouped by the plane whose identity it runs under.

Editable source SVG draw.io All views
Capture plane — namespace audit-ingest · managed identity ingest-mi envoy-gateway Envoy · mTLS · 12 pods ingest-api Rust axum · 24 pods admission schema · secrets · size quota-governor OPA · per tenant quarantine-writer rejections, audited Durable buffer — Strimzi on dedicated node pool kafka-broker 12 × D16s v5 · RF 3 kraft-controller 3 pods, 3 zones tiered-storage 6 h local, 72 h Blob Custody plane — namespace audit-custody · managed identity custody-mi sealer Go · 32 pods chain-service Go · 6, leader per shard checkpoint-signer Go · 3 pods compactor chain-preserving retention-engine Go · 3 pods disposal-service Go · 2 pods Query plane — namespace audit-query · managed identity query-mi indexer Go · 24 pods clickhouse 16 × E16ds v5 query-api Go · 12 pods reconciler Go · 4 pods trino archive, autoscaled Evidence plane — namespace audit-evidence · managed identity evidence-mi verifier Go · 16 pods proof-service Go · 6 pods export-service Go · 6 pods Azure Blob Storage immutable containers Confidential Ledger separate subscription Managed HSM per-tenant keys PostgreSQL Flexible registry, holds Valkey Cluster dedup, quota acks=all, isr 2 consume PUT once append head consume Container Architecture — Deployable Units (C4 L2) Interface / broker Application we own Queue / topic Security / platform Data store synchronous event / async Five namespaces, five managed identities, five Entra groups with no shared membership. Blob data-plane roles are granted per plane: the sealer may create a blob and no more; nothing anywhere holds Storage Blob Data Owner. v 1.0 · owner Data & AI Global Practice · date 2026-09

Decisions

  • One namespace and one managed identity per plane, with no group nesting between the Entra groups that back them. The namespace boundary is doing real authorization work, not just tidying.
  • The ingest API is Rust because the p99 budget is 150 ms including a secret scan and a Kafka round trip, and a garbage-collected tail is most of that budget.
  • The chain service is leader-elected per shard rather than singleton: 500 tenant chains extend in parallel, which is what makes 100 k events/s sealing possible at all.

Sizing

  • 24 ingest pods sustain 25 k/s steady with headroom to 100 k/s on 3× horizontal scale; Envoy terminates mTLS ahead of them.
  • 12 Kafka brokers on D16s v5, RF 3 across three zones, 240 partitions, 6 h local retention with tiered storage extending the replay window to 72 h.
  • 16 ClickHouse nodes on E16ds v5, NVMe for the hot 30 days and Blob-backed storage beyond it.

Assumptions

  • Strimzi-managed Kafka rather than Event Hubs: the durability contract needed here is acks=all with min.insync.replicas=2 and an explicitly controlled retention, and owning the broker is the cheaper way to own that contract. Event Hubs Premium remains the managed fallback if the operational load proves unwelcome.
◀  Layered Architecture All views Ingest Plane  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 40 views · Microsoft Azure and open source09-container-architecture