AI Agent Orchestration Platform  ·  View 15 of 32  ·  3 · Data

Data Flow by Data Class

Six classes of data, each traced from where it is produced to where it finally rests.

Editable source SVG draw.io All views
Produce Transport Land Serve Retain Definitions Git commit Pipeline · on merge Azure SQL Signed bundle All versions kept Execution state Orchestrator step Checkpoint · sync Cosmos DB Query and replay 90 d then archive Prompts and output Model call Gateway capture ADLS · encrypted Trace deep link Off by default Memory and knowledge Source system Ingestion · nightly AI Search index Trimmed retrieval Follows source Telemetry and cost OTel span Event Hubs Monitor + ADX Grafana · console 90 d hot · 2 y cold Audit Privileged action Append only Immutable blob Sentinel + export 7 years · WORM Data Flow by Data Class — Produce, Transport, Land, Serve, Retain Application we own Interface / broker Data store Security / platform External / third party Queue / topic Prompt and completion capture is opt-in per project and defaults off. Everything else on this page is always on. v 1.0 · owner Data Architecture · date 2026-08

Decisions

  • Prompt and completion capture is opt-in per project and defaults to off; the platform is useful without it
  • Telemetry and audit are always on and cannot be disabled by a tenant — they are what makes the platform governable
  • Definitions flow only one way, from Git through the pipeline into the registry; there is no console path that bypasses it

Cadence

  • Checkpoints are synchronous and on the critical path; telemetry is asynchronous and never is
  • Knowledge ingestion is nightly by default, with an on-demand path for a corpus that changes intra-day
  • Cost facts roll up hourly; the live budget check reads the reserved-token counter in Redis, not the rollup

Assumptions

  • Source systems can be crawled with an identity that reflects end-user ACLs, so security trimming has something to trim on
  • Legal has signed off the default retention periods per class; a tenant may shorten them but not extend beyond the organisation ceiling
◀  Context Management All views Platform Data Model  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-08 · 32 views · Microsoft Azure15-data-flow-by-class