Feature Store  ·  View 10 of 21  ·  Data

Storage Zones by Durability

The largest and hottest store has the weakest durability requirement; the smallest is the only irreplaceable one.

Editable source SVG draw.io All views
Authoritative — loss is unrecoverable Registry RPO 0, RTO 1 h Definition repo Git, reviewed Audit log Object Lock, 7 yrs Re-derivable from sources Offline store RPO 15 min, 25 mo Raw landing as given Training-set manifest kept forever Derived and disposable Online store rebuild ≤ 90 min Read cache seconds Materialised sets 13 mo, regenerable In flight and evidential Stream log 7 days, replayable Serving log 90 days Drift & skew record 13 months Source systems the real archive rebuild re-fetch regenerate Feature Store — Storage Zones by Durability Data store Queue / topic External / third party batch The hottest and largest store has the weakest durability requirement. The smallest store is the only irreplaceable one. v 1.0 · owner Data Platform Architecture · date 2026-09

Decisions

  • Four zones graded by what it would take to get the data back. Only the registry, the definition repo and the audit log are unrecoverable, and all three are small.
  • The online store — the most expensive and most read store in the platform — is explicitly disposable. That inverts its requirements: weak durability, absolute correctness.
  • A training set's manifest is kept forever and the materialised set for 13 months, because the manifest is the artefact and the data is a consequence of it.

Assumptions

  • Offline history 25 months at full grain; online store latest value only with per-feature TTL; serving log 90 days; audit 7 years.
  • Full rebuild of every online group ≤ 90 min; one group ≤ 10 min.

Risks

  • A rebuild path that is documented but not exercised is a rebuild path that does not exist. This is the design's largest operational dependency and the one most likely to rot.
◀  Feature Data Flow All views Registry and Value Data Model  ▶
Solution Architecture v1.0 · Amazon Web Services · Data Platform Architecture · 2026-09 · 21 views · 14 architecture decision records10-storage-durability-zones