Audit Log Service  ·  View 32 of 40  ·  6 · Operations

Observability

Signal type by pipeline stage, and the deliberate emptiness of the monitoring stack.

Editable source SVG draw.io All views
Ingest Buffer Seal and chain Custody Query Evidence Metrics · Prometheus ack p50 / p99 reject rate isr, lag, disk per partition Seal age unsigned interval Objects locked tier mix p95 by range rows scanned Verify duration divergence count Logs · Loki Rejection reasons never the payload Broker events Seal gaps resume points Lifecycle actions Query hash only text is in the log Verdict lines Traces · OpenTelemetry Gateway → produce Seal → sign Query → fetch Bundle assembly Alerts · Alertmanager p99 > 150 ms 5 min window isr page immediately Seal age > 120 s Delete attempted always an incident Export anomaly volume, hour, scope Divergence within 15 min Into the log itself Quota denial Topic config change Key use Disposal certificate record.read record.export Observability — Signal Type by Pipeline Stage The monitoring stack never receives record content. Metrics carry counts and latencies, logs carry reasons and hashes, and anything that would tell an operator what a record said belongs in the log rather than in Grafana. v 1.0 · owner Data & AI Global Practice · date 2026-09

Decisions

  • The monitoring stack never receives record content. Metrics carry counts and latencies; logs carry reasons and query hashes. Anything that would tell an operator what a record said belongs in the log, not in Grafana.
  • The bottom row is the point: six categories of event that the service writes about itself, into itself. Quota denials, topic configuration changes, key use, disposal certificates, reads and exports.
  • Alerts are named after a runbook, not after a threshold. An operator woken at 3 a.m. by 'seal age > 120 s' gets a page that tells them the buffer is absorbing it and what the deadline is.

The alerts that matter

  • Any attempted delete on the segment container: always an incident, never a warning.
  • Chain divergence: paged within 15 minutes of detection.
  • Export anomaly by volume, hour or scope: the realistic exfiltration signal, and the one an operator cannot investigate themselves because they cannot read records.

Assumptions

  • Detections flow to Microsoft Sentinel as an alert feed. Sentinel is a consumer of signals about this service, never a copy of it — a second searchable copy of the audit corpus outside the custody plane would defeat the access model.