Real-Time Analytics Platform  ·  View 08 of 21  ·  Data

Storage Zones, Tiering and Retention

Every store the platform owns, grouped by who owns it and whether it can be rebuilt — the view that answers what happens if a given store is lost.

Editable source SVG draw.io All views
Platform storage estate
Platform storage estate
Hot · Azure Data Explorer
Hot · Azure Data Explorer
Raw Event Table
hot cache 31 d
Raw Event Table...
Materialized Views
pre-aggregated metrics
Materialized Views...
Dimension Tables
reference lookups
Dimension Tables...
Late Arrival Table
beyond watermark
Late Arrival Table...
Warm · ADLS Gen2 Delta Lake
Warm · ADLS Gen2 Delta Lake
Bronze · Raw Capture
immutable · system of record
Bronze · Raw Capture...
Silver · Enriched
rebuildable from bronze
Silver · Enriched...
Gold · Curated Marts
rebuildable from silver
Gold · Curated Marts...
Cold · archive and export
Cold · archive and export
ADX Cold Tier
32 d to 400 d
ADX Cold Tier...
Blob Archive
7 y · legal hold
Blob Archive...
External Tables
hot + cold union query
External Tables...
Operational state
Operational state
Stream Checkpoints
ADLS · per job
Stream Checkpoints...
Schema Registry
versioned contracts
Schema Registry...
Offset & Watermark State
RocksDB + ADLS
Offset & Watermark State...
Single Unrebuildable Zone
bronze loss is terminal
Single Unrebuildable Zone...
Microsoft Purview
classification · lineage
Microsoft Purview...
Lifecycle Policy
hot to cool to archive
Lifecycle Policy...
Azure Key Vault
customer-managed keys
Azure Key Vault...
cache eviction
cache eviction
rebuild
rebuild
curate
curate
after 90 d
after 90 d
union query
union query
tier policy
tier policy
Storage Zones, Tiering and Retention
Storage Zones, Tiering and Retention
Data store
Data store
Security / platform
Security / platform
Risk / gap
Risk / gap
batch
batch
synchronous
synchronous
Retention is policy-driven per table; no data is deleted by a job, only by an expiry policy.
Retention is policy-driven per table; no data is deleted by a job, only by an expiry policy.
v 1.0 · owner Data Architecture · date 2026-08
v 1.0 · owner Data Architecture · date 2026-08
Text is not SVG - cannot display

Rebuildability

  • Bronze capture is the only zone that cannot be reconstructed from anything else
  • ADX tables, materialized views, silver and gold are all derivable from bronze
  • Losing the hot store costs a replay window, not data; losing bronze is terminal

Cost shape

  • Hot cache is the dominant cost line: 31 days at 3.5 TB/day is roughly 110 TB cached
  • Shrinking the hot window is the first cost lever and directly trades against query latency
  • Archive tier at 7 years is cheap; the cost there is rehydration time, not storage

Governance

  • Purview classifies columns; classification drives masking policy in the serving layer
  • Retention is expressed as an ADX policy and a blob lifecycle rule, never as a delete job
  • Customer-managed keys in Key Vault with 12-month rotation across every data zone