AI Agent Orchestration Platform  ·  View 12 of 32  ·  3 · Data

Data Architecture and Storage Zones

What is authoritative, what is rebuildable, what is large, and what disaster recovery actually has to cover.

Editable source SVG draw.io All views
Authoritative · loss is unrecoverable · backup and DR mandatory Configuration and governance Definition Store Azure SQL · PITR 35 d Credential Store Key Vault · soft delete Policy Bundles signed · versioned Execution truth Execution Store Cosmos · continuous BU Orchestration History durable · replayable Audit Ledger immutable blob · 7 y Rebuildable · derived, restorable by reprocessing Indexes and caches Knowledge Index AI Search · reindex Turn and Route Cache Redis · TTL 60 m Semantic Cache Redis vector · 24 h Analytical projections Telemetry Store ADX · 90 d hot Run Projections console read model Cost Facts daily rollup · 24 m Large payload · lifecycle and retention policy driven Object storage Run Artifacts ADLS · tier at 30 d Prompt and Response opt-in · 30 d Golden Datasets versioned · retained Event Capture Avro · 12 m Backup Vault GRS · immutable Paired-Region Replica RPO 5 min Microsoft Purview catalogue · lineage nightly continuous WORM copy reindex ingest scan Data Architecture — Storage Zones by Ownership and Rebuildability Data store Security / platform External / third party batch event / async Only the top zone needs disaster recovery. Everything in the middle zone is rebuilt from it, and that is the sizing basis for RTO. v 1.0 · owner Data Architecture · date 2026-08

The organising idea

  • Storage is chosen per job rather than forced into one database, and each store is placed in one of three zones by what its loss would cost
  • Only the authoritative zone needs disaster recovery; the rebuildable zone is restored by reprocessing, which is what makes the RTO achievable
  • Large payloads never sit in the transactional stores — the execution record holds a reference, not a blob

Targets

  • RPO 5 minutes and RTO 15 minutes for execution state; RPO effectively zero for definitions via zone-redundant SQL with a geo-replica
  • Traces 90 days hot and 2 years archived; audit 7 years write-once; prompts and completions 30 days when capture is enabled
  • Index rebuild from source measured at under 4 hours for the full knowledge corpus

Risks

  • Rebuild time is only credible if it is exercised; a quarterly restore drill is a delivery commitment, not an aspiration
  • Retention is policy-driven per tenant, so a misconfigured policy is a compliance event — retention changes require approval and are audited
◀  Definition Lifecycle and Versioning All views Memory Subsystem  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-08 · 32 views · Microsoft Azure12-data-architecture-zones