Audit Log Service  ·  View 18 of 40  ·  4 · Data

Retention and Tiering

What each store does at each age — and the one property that does not change as things get cheaper.

Editable source SVG draw.io All views
0 – 30 days 30 days – 13 months 13 months – 7 years 7 – 10 years (elevated) After expiry Segment store Blob Hot ZRS locked policy Blob Cool ZRS locked policy Blob Archive locked policy Blob Archive elevated class only Crypto-Shred certificate signed Query path ClickHouse NVMe p95 2 s ClickHouse on Blob per-tenant opt-in Trino over segments async, ≤ 12 h Trino over segments async, ≤ 12 h Rows Dropped index only Checkpoint ledger Confidential Ledger append only Confidential Ledger append only Confidential Ledger append only Confidential Ledger append only Outlives Records proofs still verify Ingest buffer Kafka · 72 h replay window Storage cost ≈ $23 / TB-month ≈ $10 / TB-month ≈ $2 / TB-month ≈ $2 / TB-month $0 and provably so Retention and Tiering — Tier Is a Cost Decision, Never an Immutability One Every cell in the top row carries the same locked retention policy. The tier changes what the storage costs and how long a read takes; it never changes what a principal is permitted to do to the object. v 1.0 · owner Data & AI Global Practice · date 2026-09

Decisions

  • Tier is a cost decision, never an immutability decision. Every cell in the segment row carries the same locked retention policy whether it is on hot, cool or archive storage.
  • The index window is tunable per tenant, independently of the record retention period, and defaults to 90 days rather than 13 months. That default is the single largest cost lever in the design and is discussed on view 33.
  • Beyond the index window, queries are served by Trino directly over the same Parquet segments — slower, asynchronous, and answering exactly the same question.

Retention classes

  • Standard: 13 months online plus 7 years archived. Elevated: 10 years. Both extendable, never reducible.
  • Disposal within 30 days of expiry when no hold applies. Legal hold suspends it indefinitely.
  • The ingest buffer holds 72 h, which is the replay window for an index rebuild or a downstream outage.

Assumptions

  • Storage prices are indicative West Europe list prices for ZRS at the time of writing and are used for relative reasoning, not for a quotation.
  • Archive-tier rehydration latency is inside the 12 h asynchronous retrieval SLA. High-priority rehydration is available at extra cost and is not budgeted.
◀  Data Flow — Commit to Answerable All views Where the Tenant Boundary Sits  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 40 views · Microsoft Azure and open source18-retention-and-tiering