Storage Tiering Service  ·  View 12 of 31  ·  4 · Data

Four Kinds of State

The catalogue, the telemetry, the records and the policy, kept in separate stores because they fail, recover and matter differently.

Editable source SVG draw.io All views
Catalogue · loss stops reads · RPO 0 Placement rows Vitess · 9.2 B Movement rows in-flight state Tenant guard rows hold epoch Telemetry · derived · RPO 5 min Raw access events ClickHouse · 90 d Access aggregates object · cohort · 3 y Canary heartbeats freshness Records · audit grade · RPO 60 s Decision records inputs · odds Movement outcomes bytes · requests · $ Parquet archive Object Lock · 3 y Policy · change-controlled Ladder versions Git · signed Policy versions CUE · rings Holds · pins · budgets PostgreSQL · dual auth Tier scan location back, history not Four Kinds of State, Four Recovery Obligations Data store Security / platform Risk / gap Each box can be lost without taking another with it. Only the first stops reads; the tier scan below rebuilds its locations, and only the third cannot be rebuilt at all. v 1.0 · owner Storage Platform Architecture · date 2026-09

Decisions

  • The catalogue is the only store whose loss stops reads. It holds where, never why, which keeps rows small and failover simple.
  • Telemetry is disposable beyond its aggregates. Losing an hour of events degrades a future decision; the classifier's freshness gate makes sure it cannot cause a wrong one (ADR-18).
  • Decision and movement records are the platform's evidence, retained for three years in Object-Locked Parquet independently of the catalogue. They cannot be rebuilt from anything, which is why they are not stored next to the thing a rebuild replaces.

Recovery objectives

  • Catalogue: RPO 0 and RTO 60 s in DC-A, RPO 10 s and RTO 15 min across sites, full rebuild by tier scan within 30 h.
  • Records: RPO 60 s, RTO 4 h. Telemetry: RPO 5 min, RTO 4 h. Recall job state: RPO 30 s, RTO 10 min.

What a rebuild loses

  • A catalogue rebuilt from storage knows where every object is, what it weighs and which pack holds it. It does not know why it moved, when it was last read or which policy put it there. Those come back from the records store, not from the scan.
◀  Anatomy of a Read All views Catalogue, Records and Policy: Data Model  ▶
Solution Architecture v1.0 · Storage Platform Architecture · 2026-09 · 31 views · open source, fully on-premises12-four-state-classes