Event-Driven Notification Platform · View 21 of 26 · 5 · Operations

Observability and Operations

Every signal, where it goes, and how a support engineer answers the question in one place.

Editable source SVG draw.io All views

The question this has to answer

Why did customer X not receive notification Y — answerable from one query, target under two minutes
It works because event_id, notification_id, provider_ref and the suppression reason are on the same trace and the same ClickHouse row
A suppressed notification is a first-class record with a reason, so the answer is never an absence of data

Signals and cost control

Tail sampling keeps every error trace and 1 percent of successes, which is what makes tracing affordable at this volume
Logs are redacted at the collector, so PII cannot reach Loki even if a service logs it
Per-tenant counters drive both the chargeback report and quota enforcement from the same series

Alerting

Pages are on SLO burn rate and on P0 DLQ depth above zero; everything else is a ticket
KEDA scales out on consumer lag before the lag alert fires, so a page means scaling did not help
Every alert carries a runbook link; an alert without one fails the release gate

◀ Autoscaling, Capacity and Backpressure All views Administration and Operations Surface ▶