Incident Management Platform  ·  View 14 of 34  ·  4 · Data

Four Stores, Separated by What May Be Lost and What Must Be Fast

The requirement's four stores plus the evidence that feeds reporting, arranged by recovery obligation rather than by technology.

Editable source SVG draw.io All views
Platform state, by recovery obligation Coverage · RPO 0 Coverage snapshot JetStream KV · R3 Escalation policies KV · versioned Schedule store PostgreSQL · source Event log · system of record · 7 y Paging outbox JetStream · 30 d Incident event log PostgreSQL · append-only Chain anchors Ceph · Object Lock Notification ledger · 13 mo Ledger stream JetStream · 30 d Ledger archive ClickHouse · 13 mo Operational · rebuildable Projections PostgreSQL schema Review search full-text index Evidence · loss-tolerant Raw payloads ClickHouse · 90 d Noise and cost marts ClickHouse · 13 mo Four Stores, Separated by What May Be Lost and What Must Be Fast The first column is small and must be current. The second must never change. The third must survive the second. The last two may be rebuilt or lost. v 1.0 · owner Reliability Architecture · date 2026-09

Decisions

  • Coverage lives in the paging domain as JetStream key-value buckets, written with quorum so a publish is RPO 0, and read locally in every cell. It is rebuildable from the schedule store, but not in the two minutes a page has, which is why it is replicated rather than recomputed.
  • The event log is the system of record and the incident record is a projection of it. Paging-side events land first in a 30-day JetStream outbox, so the log can be unavailable for a month without losing a paging event (ADR-08).
  • The notification ledger is written by the dispatcher, not by the incident service. The record of what was sent must not depend on the thing it might be evidence against.

Retention

  • Incident events and audit: 7 years, immutable, with daily hash-chain anchors in an Object Lock bucket. Notification ledger: 13 months. Raw payloads: 90 days. Projections: none, rebuilt on demand.

Risks

  • Projections rebuilt from seven years of events can be slow exactly when the console is being restored. The rebuild is timed quarterly against the 60-minute target, and projections for open incidents are rebuilt first.
◀  May the Platform Decide Not to Page? All views Incidents, Coverage and Evidence — Data Model  ▶
Solution Architecture v1.0 · Reliability Architecture · 2026-09 · 34 views · open source, fully on-premises14-four-stores