AI Agent Orchestration Platform

Architecture Views

32 views, in reading order. Every view ships three ways: an HTML page, an SVG that re-opens in diagrams.net fully editable, and draw.io source.

A centralised runtime and control plane for designing, deploying, orchestrating, observing and governing AI agents and multi-agent workflows on Azure. Read the set in order: acts 1 and 2 fix the boundary and the structure, act 3 says who owns which data, act 4 shows what happens at runtime, act 5 covers how it is run, act 6 covers why it is safe and what is delivered when. Each page carries the diagram, the decisions behind it, and links to the editable SVG and draw.io source.

1 · Context and scope

The boundary, the actors, the dependencies, and the plane separation that governs every page after it.
01 Identity and governance Microsoft Entra ID SSO · OIDC · agent ID Microsoft Purview classification Microsoft Sentinel SIEM People Agent Developer SDK · CLI · builder Business User chat · app · API Approver Teams card Platform Admin console SRE on call Model providers Azure AI Foundry GPT · embeddings Foundry Models partner catalogue Open-Weight Models AKS · KAITO External LLM APIs egress controlled AI Agent Orchestration Platform Azure · control + execution Tools and systems of record MCP Tool Servers internal + partner SaaS Applications SAP · SFDC · SNOW Enterprise APIs via APIM Content Estate SharePoint · ADLS Microsoft Teams Graph tokens labels security events defines agents runs agents approves actions sets policy operates inference specialist routes sovereign routes fallback only tool calls actions read and write grounding notifications AI Agent Orchestration Platform — System Context External / third party Person or role synchronous event / async Model and tool endpoints are reached only through the platform gateway. No agent holds a provider key. v 1.0 · owner Data & AI Global Practice · date 2026-08 System Context Who uses the platform, what it depends on, and where the boundary is drawn. HTML page SVG draw.io
02 Entry Front Door WAF · anycast Platform API APIM · OIDC SDK, CLI, Console Python · .NET · TS Admission Admission Service resolve and pin Policy Engine OPA · signed bundle Budget Guard cost and token cap Orchestration Durable Orchestrator Durable Functions Scheduler priority classes Agent Coordinator multi-agent patterns Approval Service external events Agent runtime Agent Executor Container Apps Context Manager assemble · compress Guardrail Chain Content Safety Session Sandbox dynamic sessions Model and tool access AI Gateway APIM GenAI policy Tool Broker MCP · REST Memory Service scoped retrieval Evidence Execution Store Cosmos DB Trace and Cost Monitor · ADX Audit Ledger immutable blob checkpoint spans decision AI Agent Orchestration Platform — High-Level Architecture Interface / broker Application we own Security / platform Data store synchronous event / async One picture for the reader who sees only one. Tenancy, regions and failure paths are later views. v 1.0 · owner Data & AI Global Practice · date 2026-08 High-Level Architecture The shape of the platform in one picture, for the reader who will see only one. HTML page SVG draw.io
03 Control Plane · what a run is allowed to be Registries Agent Registry versioned specs Workflow Registry published graphs Tool Registry API Center Model Registry routes and quotas Governance Policy Authoring Rego · versioned Identity and RBAC Entra · scopes Credential Broker Key Vault refs Config Promotion App Configuration Control data Definition Store Azure SQL · HA Bundle Publisher signed · immutable Execution Plane · what a run actually does Orchestration Durable Orchestrator replay-safe Scheduler queue classes Work Queues Service Bus Runtime Runtime Workers stateless · KEDA Policy Sidecar local decisions Session Sandbox Hyper-V isolated Execution data Execution Store Cosmos · partitioned Turn Cache Managed Redis Artifact Store ADLS Gen2 AI Gateway and Tool Broker APIM · shared Observability and Cost Plane Monitor · ADX signed bundle scoped token checkpoint dispatch gateway telemetry Control Plane and Execution Plane — The Governing Boundary Application we own Security / platform Data store Interface / broker Queue / topic event / async synchronous A control-plane outage stops new admissions. Runs already admitted carry a pinned bundle and continue. v 1.0 · owner Data & AI Global Practice · date 2026-08 Control Plane and Execution Plane The one architectural boundary that everything else on this site depends on. HTML page SVG draw.io

2 · Structure

The layering rule, the deployable units, every interface, and the three planes agents actually touch: model, tool and message.
04 Experience Agent Builder Workflow Designer Admin Console SDK and CLI Chat and App Channels Platform API API Management AuthN and RBAC Streaming Endpoint Webhook Ingress Control plane Agent Registry Workflow Registry Tool Registry Model Registry Policy Service Prompt and Config Orchestration Durable Orchestrator Scheduler Agent Coordinator Event Manager Approval Service State Manager Agent runtime Agent Executor Context Manager Guardrail Chain Tool Executor Memory Service Checkpointer Sandbox Access AI Gateway Tool Broker Retrieval Service Credential Broker Data Azure SQL Cosmos DB Managed Redis Azure AI Search ADLS Gen2 Service Bus Key Vault Assurance Azure Monitor Data Explorer Managed Grafana Evaluation Service Audit Ledger Microsoft Sentinel HTTPS REST submit run dispatch inference tool call meter state spans promotion gate Layered Architecture — What Depends on What Application we own Interface / broker Security / platform Data store Queue / topic synchronous event / async batch Dependencies point downward. The one exception is drawn: evaluation scores feed the promotion gate in the control plane. v 1.0 · owner Data & AI Global Practice · date 2026-08 Layered Architecture What depends on what, and the single dependency that points the wrong way. HTML page SVG draw.io
05 Subscription: agentplat-prod · VNet 10.40.0.0/16 Edge subnet Front Door + WAF Premium · Private Link API Management Premium · internal VNet Control plane subnet · AKS (3 AZs) Registry API .NET 9 · 6 pods Policy Service OPA · 4 pods Config Service App Config · 3 pods Credential Broker workload identity Console BFF TypeScript · 4 pods Execution plane · Container Apps environment Durable Orchestrator Functions · 10-80 Scheduler leader-elected · 3 Runtime Workers Python · KEDA 20-400 Tool Executor Go · KEDA 10-120 Approval Service .NET 9 · 3 pods Memory Service Python · 6 pods Session pool · Hyper-V isolated, no default egress Code Sessions dynamic sessions Browser Sessions computer use Data subnet · private endpoints only Azure SQL Business Critical Cosmos DB session consistency Managed Redis Balanced B5 Azure AI Search S2 · 3 replicas ADLS Gen2 ZRS · lifecycle Service Bus Premium · 4 MU Key Vault HSM · RBAC Azure AI Foundry PTU + PAYG MCP Tool Servers internal + partner SaaS Applications SAP · SFDC · SNOW HTTPS REST submit run dispatch exec request gateway hop inference MCP REST checkpoint Container and Component Architecture — Azure Deployable Units Interface / broker Application we own Security / platform Data store Queue / topic External / third party synchronous event / async Replica ranges are the sizing basis for 5,000 concurrent executions. Every data service is reached over a private endpoint. v 1.0 · owner Data & AI Global Practice · date 2026-08 Container and Component Architecture The deployable units, their technology, their replica ranges and the Azure service each one owns. HTML page SVG draw.io
06 Inbound Chat Channels Business Applications Event Sources Partner Webhooks CI/CD Pipelines Platform edge Front Door + WAF TLS 1.3 · anycast API Management 38 public operations Event Grid 6 published topics Scheduled Triggers cron · per project Platform Core control + execution Outbound Azure AI Foundry Foundry Models MCP Tool Servers Enterprise APIs SaaS Applications Microsoft Graph Microsoft Purview Microsoft Sentinel SSE · realtime REST · realtime CloudEvents HMAC signed OIDC · deploy inference · sync inference · sync MCP · on demand REST · on demand REST · on demand cards · events lineage · nightly security events Integration and Interface Catalogue External / third party Interface / broker Queue / topic Application we own synchronous event / async batch Every interface is owned by the platform except the MCP wire contract and the Graph card schema. v 1.0 · owner Integration Architecture · date 2026-08 Integration and Interface Catalogue Every way something enters or leaves the platform, with protocol, direction and cadence. HTML page SVG draw.io
07 Lease Queue Lease session locked Idempotency Check step key Bundle Load pinned version Prepare Context Manager assemble window Memory Read scoped · audited Limit Check tokens · depth · cost Guard in Prompt Shields injection · jailbreak PII Redaction detect · mask Classification Gate residency route Reason Agent Loop framework adapter AI Gateway Call stream · retries Plan or Delegate coordinator hop Act Tool Call Validation schema · permission Tool Executor timeout · retry Session Sandbox code · browser Guard out Structured Output schema enforced Groundedness Check citation bound Escalate to Human on fail Commit Checkpoint durable state Memory Write policy TTL Emit Telemetry OTel GenAI awaits tool result Agent Runtime — Inside One Worker Queue / topic Decision point Application we own Security / platform Interface / broker Data store event / async synchronous The worker holds no long-lived secret and no state between steps. Everything durable is written at Commit. v 1.0 · owner Runtime Engineering · date 2026-08 Agent Runtime Internals What one worker does between leasing a step and committing it, and where the controls sit. HTML page SVG draw.io
08 AI Gateway · Azure API Management Request policy chain Identity Check agent identity Token Rate Limit per tenant · TPM Semantic Cache Redis · vector Egress Guard classification Router Capability Match modality · context Route Policy cost · latency · region Weighted Pool load balanced Circuit Breaker 429 · 5xx trip Response policy chain Token Metric Emit per execution Stream Relay SSE passthrough Fallback Chain next healthy route Model backends Azure AI Foundry Reasoning PTU Pool reserved capacity PAYG Spillover burst · 3 regions Embedding Models batch friendly Alternate providers Foundry Models partner catalogue Open-Weight on AKS KAITO · GPU pool Reranker self-hosted Model Registry control plane Provider Health probe · 30 s Cost Meter Event Hubs · ADX route table backend state primary on trip on trip tokens Model Abstraction — AI Gateway, Routing and Fallback Security / platform Data store Decision point Interface / broker External / third party Application we own event / async synchronous failure / alternate No agent holds a provider key. The gateway presents its own managed identity to every backend. v 1.0 · owner Data & AI Global Practice · date 2026-08 Model Abstraction and Routing How a request for a capability becomes a call to a specific model, and what happens when that model is unavailable. HTML page SVG draw.io
09 Discover Authorise Invoke Validate Record HTTP and REST API Center entry Agent scope grant APIM egress JSON Schema Tool call span MCP servers MCP tools/list Server allow-list MCP over HTTP Result schema Tool call span Data and search Index catalogue Row-level filter AI Search query Citation binding Provenance record Code and browser Session template Egress policy Dynamic session Output size cap Artifact to ADLS Sensitive actions Marked in registry Human approval Idempotency key Effect assertion Audit ledger Tool Plane — Discovery, Authorisation and Invocation by Tool Class Application we own Security / platform Interface / broker Data store Decision point Every lane ends in a record. A tool the registry does not list cannot be called, whatever the model asks for. v 1.0 · owner Integration Architecture · date 2026-08 Tool Plane and Permissions How each class of tool is discovered, authorised, invoked, validated and recorded. HTML page SVG draw.io
10 Azure Service Bus Premium · work distribution Work queue classes Interactive p95 lease under 2 s Batch throughput first Long-Running hours to days Retry Scheduled backoff + jitter Delivery controls Session Ordering per execution Lock Renewal heartbeat Dead Letter per queue Azure Event Grid · lifecycle and webhooks Published topics Agent Lifecycle start · pause · end Workflow Events state transitions Approval Events raised · decided Guardrail Events block · escalate External edges Webhook Ingress HMAC · replay guard Partner Delivery retry 24 h Azure Event Hubs · telemetry firehose Streams Trace Stream OTel GenAI Token Meter per model call Capture to ADLS Avro · 5 min DLQ Triage Worker classify · replay Operator Alert depth and age SLO poison message replay depth breach notify capture Event and Messaging Topology Queue / topic Interface / broker Data store Application we own Security / platform failure / alternate event / async batch Delivery is at-least-once everywhere. Effects are made idempotent at the handler, never assumed at the broker. v 1.0 · owner Platform Engineering · date 2026-08 Event and Messaging Topology Which queues and topics exist, why there are several, and how a stuck message reaches a human. HTML page SVG draw.io
11 Author Agent Spec YAML · typed schema Local Runtime same contract Git Repository source of truth Validate Schema Lint spec version Policy Simulation dry run Golden Evaluation regression suite Review Behaviour Diff vs live version Owner Approval RBAC gated Risk Classification tool blast radius Publish Immutable Version content hash Bundle Signing Key Vault key Registry Entry draft to published Promote Development auto Test and Staging eval gate Production Canary 5 percent · 24 h Operate Version Pinning at admission A/B Split traffic weight Rollback prior hash Retire drain in-flight scores canary fails resolve hash Definition Lifecycle — Versioning an Agent, Workflow, Prompt or Policy Application we own Data store Security / platform Decision point synchronous failure / alternate A published version is immutable. Change means a new version; rollback means re-pointing the alias, never editing. v 1.0 · owner Data & AI Global Practice · date 2026-08 Definition Lifecycle and Versioning How an agent, workflow, prompt, tool or policy definition becomes an immutable published version. HTML page SVG draw.io

3 · Data

Which store holds what, how memory is scoped, how a context window is assembled, and what a decision reads.
12 Authoritative · loss is unrecoverable · backup and DR mandatory Configuration and governance Definition Store Azure SQL · PITR 35 d Credential Store Key Vault · soft delete Policy Bundles signed · versioned Execution truth Execution Store Cosmos · continuous BU Orchestration History durable · replayable Audit Ledger immutable blob · 7 y Rebuildable · derived, restorable by reprocessing Indexes and caches Knowledge Index AI Search · reindex Turn and Route Cache Redis · TTL 60 m Semantic Cache Redis vector · 24 h Analytical projections Telemetry Store ADX · 90 d hot Run Projections console read model Cost Facts daily rollup · 24 m Large payload · lifecycle and retention policy driven Object storage Run Artifacts ADLS · tier at 30 d Prompt and Response opt-in · 30 d Golden Datasets versioned · retained Event Capture Avro · 12 m Backup Vault GRS · immutable Paired-Region Replica RPO 5 min Microsoft Purview catalogue · lineage nightly continuous WORM copy reindex ingest scan Data Architecture — Storage Zones by Ownership and Rebuildability Data store Security / platform External / third party batch event / async Only the top zone needs disaster recovery. Everything in the middle zone is rebuilt from it, and that is the sizing basis for RTO. v 1.0 · owner Data Architecture · date 2026-08 Data Architecture and Storage Zones What is authoritative, what is rebuildable, what is large, and what disaster recovery actually has to cover. HTML page SVG draw.io
13 Written by Store Retrieval Permission Retention Execution state Orchestrator Cosmos DB By execution id Run owner only 90 days Conversation Agent Executor Cosmos DB Recent turns Session subject Tenant policy Episodic Memory Service Cosmos + vectors Similarity + recency Agent scope grant Decay then expire Knowledge Ingestion Pipeline Azure AI Search Hybrid + reranker Security trimming Source lifecycle Profile and org Admin and user Azure SQL Direct lookup Consent flagged Erasure on request Artifacts Tool Executor ADLS Gen2 By artifact id SAS · 15 minutes Tier then delete Memory Subsystem — Scope, Store, Retrieval, Permission, Retention Application we own Data store Interface / broker Security / platform Person or role Every read is filtered by tenant and by the agent's declared memory scope, and every read is written to the audit ledger. v 1.0 · owner Data Architecture · date 2026-08 Memory Subsystem Six kinds of memory, where each lives, how it is retrieved, who may read it and how long it survives. HTML page SVG draw.io
14 Resolve Context Strategy per agent spec Window Budget reserve for output Gather System Instruction pinned version Conversation Slice last N turns Memory Recall scoped query Knowledge Retrieval hybrid search Rank Reranker cross-encoder Deduplicate near-duplicate drop Relevance Cut score threshold Compress Rolling Summary cheap model Tool Output Trim head and tail Fit to Window hard cap Protect PII Redaction reversible token Untrusted Fencing content delimiters Classification Check route constraint Assemble Message Array provider neutral Provenance Tags source per block Token Accounting pre-call estimate still over budget route pinned cited blocks Context Management — Assembling One Model Invocation Application we own Decision point Data store Security / platform failure / alternate synchronous Retrieved content is fenced as untrusted before it reaches the model. Provenance survives into the trace, so any claim is attributable. v 1.0 · owner Data & AI Global Practice · date 2026-08 Context Management How one model invocation's context window is assembled, compressed, protected and accounted for. HTML page SVG draw.io
15 Produce Transport Land Serve Retain Definitions Git commit Pipeline · on merge Azure SQL Signed bundle All versions kept Execution state Orchestrator step Checkpoint · sync Cosmos DB Query and replay 90 d then archive Prompts and output Model call Gateway capture ADLS · encrypted Trace deep link Off by default Memory and knowledge Source system Ingestion · nightly AI Search index Trimmed retrieval Follows source Telemetry and cost OTel span Event Hubs Monitor + ADX Grafana · console 90 d hot · 2 y cold Audit Privileged action Append only Immutable blob Sentinel + export 7 years · WORM Data Flow by Data Class — Produce, Transport, Land, Serve, Retain Application we own Interface / broker Data store Security / platform External / third party Queue / topic Prompt and completion capture is opt-in per project and defaults off. Everything else on this page is always on. v 1.0 · owner Data Architecture · date 2026-08 Data Flow by Data Class Six classes of data, each traced from where it is produced to where it finally rests. HTML page SVG draw.io
16 tenant tenant_id PK entra_tenant UQ residency_region quota_profile budget_ceiling project project_id PK tenant_id FK environment monthly_budget capture_prompts agent_version agent_version_id PK project_id FK spec_hash UQ state model_route_id FK workflow_version workflow_version_id PK graph_hash UQ pattern state compat_range policy policy_id PK scope_level scope_ref bundle_hash effective_from execution execution_id PK project_id FK workflow_version_id FK status correlation_id step step_id PK execution_id FK agent_version_id FK attempt checkpoint_ref tool_binding binding_id PK tool_version_id agent_version_id FK sensitivity credential_ref memory_item memory_id PK scope_type scope_ref embedding_ref expires_at cost_record cost_id PK execution_id FK prompt_tokens completion_tokens amount_usd tool_call tool_call_id PK step_id FK idempotency_key UQ outcome duration_ms approval approval_id PK binding_id FK step_id FK decision decided_by 1 : N 1 : N 1 : N N : M 1 : N 1 : N 1 : N 1 : N 1 : N 1 : N governs Platform Data Model — Core Entities tenant_id is the partition key on every execution-plane container. No entity is reachable without it. v 1.0 · owner Data Architecture · date 2026-08 Platform Data Model The entities a governance question actually reads: who ran what, on which version, with whose approval, at what cost. HTML page SVG draw.io

4 · Runtime

One run end to end, six coordination patterns, durable recovery, human approval, and the guardrail and policy chain.
17 Caller API Management Admission Service Durable Orchestrator Runtime Worker AI Gateway Tool Broker Execution Store 1. POST /v1/executions 2. verify token, scope 3. submit run 4. pin versions, evaluate policy 5. 429 budget exhausted 6. start orchestration 7. checkpoint: admitted 8. dispatch step 9. assemble context, guard input 10. chat completion, stream 11. token stream 12. tool call requested 13. invoke with idempotency key 14. tool result 15. continue with result 16. final answer 17. guard output, enforce schema 18. checkpoint: completed 19. 200 result and cost Execute an Agent — Critical Flow with Streaming Tokens stream from the gateway to the caller while the run is still open. The result is not final until the checkpoint commits. v 1.0 · owner Runtime Engineering · date 2026-08 Execute an Agent One synchronous run end to end, including the tool hop and the point at which the result becomes final. HTML page SVG draw.io
18 Trigger Decompose Delegate Execute Converge Supervisor and worker Single goal Supervisor splits Fan-out to workers Isolated contexts Supervisor merges Planner and executor Open task Plan as DAG Steps to queue Executor per step Replan on failure Hierarchical teams Programme goal Team leads Depth limit 3 Team sub-runs Roll up by level Peer collaboration Shared brief No fixed owner Message passing Shared scratchpad Quorum or timeout Debate and critic Draft answer Critic prompt Round cap 3 Revise and rescore Best scored wins Specialist routing Ambiguous intent Capability match Registry lookup One specialist Return or reroute Multi-Agent Coordination — Six Patterns on One Runtime Application we own Interface / broker Decision point Queue / topic Security / platform Data store All six run on the same orchestrator and the same worker image. The pattern is a workflow definition, not a different platform. v 1.0 · owner Data & AI Global Practice · date 2026-08 Multi-Agent Coordination Six coordination patterns, and the claim that all six are workflow definitions rather than different platforms. HTML page SVG draw.io
19 Progress Step Completes worker result Checkpoint Write before ack Queue Ack lock released Failure Process Crash pod evicted Node or Zone Loss AZ outage Provider Outage model 5xx Detect Lock Expiry no renewal Heartbeat Miss orchestrator Classify Failure eight classes Recover Redeliver Message same session Idempotency Guard step key seen Scheduled Retry backoff + jitter Provider Failover next route Resume Load Checkpoint last committed Pinned Definition original hash Continue Step no re-execution Exhausted Dead Letter after max attempts Compensate reverse effects Human Escalation with full trace transient provider permanent run continues on partial reversal Durable Execution — Checkpoint, Failure, Recovery and Replay Application we own Data store Queue / topic Risk / gap Interface / broker Security / platform Decision point Person or role synchronous failure / alternate The checkpoint is written before the queue acknowledgement, so a crash duplicates work rather than losing it. v 1.0 · owner Runtime Engineering · date 2026-08 Durable Execution and Replay What survives a crash, how failure is classified, and where a run goes when recovery is exhausted. HTML page SVG draw.io
20 Runtime Worker Policy Engine Approval Service Durable Orchestrator Teams via Graph Approver Audit Ledger 1. may I call this tool 2. approval required 3. raise request with evidence 4. record raised 5. suspend, await event 6. adaptive card to queue 7. notify approver role 8. approve or reject 9. verify role, not requester 10. record decision and actor 11. raise external event 12. resume from checkpoint 13. SLA expired, auto-reject 14. terminal state Human-in-the-Loop — Approving a Sensitive Action The approver cannot be the requester. An unanswered request auto-rejects at the configured SLA and the run ends cleanly. v 1.0 · owner Data & AI Global Practice · date 2026-08 Human-in-the-Loop How a run suspends for a human decision, and what happens when nobody answers. HTML page SVG draw.io
21 Detect Decide Act Record Prompt injection Prompt Shields Severity threshold Strip and refuse Guardrail event Sensitive data egress PII and label scan Classification rule Redact or reroute Audit with hash Unsafe tool use Argument inspection Permission and blast Block or escalate Audit ledger Ungrounded output Groundedness check Citation coverage Retry then caveat Quality metric Untrusted content Source trust tier Fence or drop No tool authority Provenance tag Harmful content Content Safety Category severity Refuse and log Sentinel signal Safety Guardrails — Detect, Decide, Act, Record by Threat Class Security / platform Decision point Application we own Queue / topic Data store Content retrieved by a tool never gains the caller's authority. Fencing happens before the model sees it, not after. v 1.0 · owner Security Architecture · date 2026-08 Safety Guardrails Six threat classes, and what the platform does about each at detect, decide, act and record. HTML page SVG draw.io
22 At admission Before model call Before tool call Before response At completion Organisation Allowed regions Approved providers Banned tool classes Content restrictions Retention policy Project Monthly budget Model allow-list Tool allow-list Capture opt-in Cost attribution Agent Run concurrency Token ceiling Granted scopes Output schema Per-agent spend Workflow Max duration Recursion depth Approval steps Compensation rule Run cost record Data classification Residency pinning Route constraint Egress denial Redaction rule Deletion trigger Policy and Budget Enforcement — Scope against Decision Point Security / platform Decision point Data store Application we own One Rego bundle, five evaluation points. Decisions are local to the worker, so a control-plane outage cannot fail them open. v 1.0 · owner Governance · date 2026-08 Policy and Budget Enforcement Five policy scopes against five decision points: exactly what is evaluated, and where. HTML page SVG draw.io

5 · Operations

Where it runs, how a change reaches production, what is watched, what it costs, and how quality improves.
23 Global · anycast and tenant-wide services Edge and identity Front Door Premium WAF · health probe Microsoft Entra ID global tenant Private DNS Zones hub linked Container Registry geo-replicated Primary region · West Europe · active Zones 1-3 · compute API Management Premium · 3 units AKS Control Plane 3 AZ node pools Container Apps Env workload profiles GPU Node Pool KAITO · spot mix Zones 1-3 · state Cosmos DB single write region Azure SQL Business Critical Service Bus Premium · ZR Managed Redis zone redundant ADLS Gen2 ZRS Secondary region · North Europe · warm standby Control plane standby API Management multi-region unit AKS Standby min replicas SQL Geo-Replica readable secondary Execution and data standby Cosmos Read Region failover priority 2 ADLS RA-GRS read access Container Apps Env scaled to zero Bastion and Jump Box no public SSH Azure Firewall Premium egress FQDN rules Azure Monitor workspace per region primary origin on failure replication auto-failover group controlled egress RA-GRS Deployment Topology — Regions, Zones and Failure Domains Interface / broker Security / platform Data store Application we own Queue / topic synchronous failure / alternate event / async Executions are region-pinned for residency. Failover replays from the last checkpoint in the surviving region; it does not migrate a live run. v 1.0 · owner Platform Engineering · date 2026-08 Deployment Topology What runs where, what is redundant, and what a region loss actually costs. HTML page SVG draw.io
24 Source GitHub Repository platform + agents Pull Request two reviewers Bicep Modules infrastructure Build Container Build reproducible SBOM and Scan Defender · Trivy Image Signing Notation · KV key Verify Unit and Contract schema fixtures Policy Simulation Rego unit tests Golden Evaluation regression gate Red Team Suite injection probes Gate Development auto on merge Test eval score gate Staging load and soak Change Approval owner + security Deploy What-If Plan drift check Blue/Green Revision Container Apps Canary Weight 5 to 50 to 100 Watch SLO Watch 24 h window Online Quality success · escalation Auto Rollback prior revision score gate burn rate breach quality regression each step CI/CD and Environment Promotion Data store Decision point Application we own Security / platform synchronous failure / alternate Agent definitions travel the same pipeline as code. A prompt change is a versioned deployment, not a console edit. v 1.0 · owner Platform Engineering · date 2026-08 CI/CD and Environment Promotion How a change reaches production, and what stops a bad one at each hop. HTML page SVG draw.io
25 Emit Collect Store Consume Act Execution traces OTel GenAI spans OTel Collector Application Insights Execution graph Failure analysis Metrics and SLO Runtime counters Managed Prometheus Monitor workspace Managed Grafana Page on burn rate Logs Structured JSON Container insights Log Analytics KQL incident search Runbook link Token and cost Gateway metric Event Hubs Data Explorer Cost breakdown Budget alert Guardrail decisions Decision event Event Grid Audit ledger Violation report Sentinel rule Quality signals Judge and feedback Evaluation runner Eval results store Agent scorecard Block promotion Observability — Signal against Pipeline Stage Application we own Interface / broker Data store Security / platform Queue / topic Decision point Execution id, workflow id, agent id and correlation id are on every row, so one identifier joins a trace to its cost and its guardrail decisions. v 1.0 · owner SRE · date 2026-08 Observability and Tracing Six signal types across emit, collect, store, consume and act — and the identifier that joins them. HTML page SVG draw.io
26 Meter Token Counting gateway policy Tool Call Units priced per API Infrastructure Cost Cost Management Attribute Execution Tagging tenant · project Rollup Job hourly · ADX Shared Cost Split by token share Enforce Pre-Call Estimate reserve tokens Soft Limit warn at 80 percent Hard Limit block at ceiling Optimise Cost-Aware Routing cheapest passing Cache Hit Rate semantic + exact PTU Utilisation reserve vs burst Context Compression tokens per run Report Cost Dashboard per tenant · agent Forecast run-rate to month end Chargeback Export monthly · CSV degrade first raise threshold avoided tokens Cost Management — Meter, Attribute, Enforce, Optimise Interface / broker Security / platform Application we own Decision point Data store failure / alternate synchronous A breached budget degrades the route before it blocks the run. Blocking is the last step, not the first. v 1.0 · owner FinOps and Architecture · date 2026-08 Cost Management and Attribution How every token and tool call is metered, attributed, capped and then made cheaper. HTML page SVG draw.io
27 Capture production traces Curate golden datasets Evaluate Offline assertions + judge Gate promotion score Release Canary 5 percent traffic Evaluate Online live scoring Improve prompt · model · tool Agent Quality target 92 percent failed and escalated runs labelled cases scores per metric pass or block canary cohort live signal new candidate version Evaluation and Quality — The Loop That Closes Data store Application we own Decision point Security / platform Metrics tracked each turn: task success, groundedness, tool-call correctness, latency, tokens, cost, failure and human escalation rate. v 1.0 · owner Data & AI Global Practice · date 2026-08 Evaluation and Quality The loop from production failure back to a better agent version, and the gate that closes it. HTML page SVG draw.io

6 · Assurance and delivery

Trust boundaries, delegated identity, tenant isolation, every failure mode, and the order of delivery.
28 Untrusted · internet Business User Partner Webhook HMAC signed Injected Content in retrieved page Credential Attack token replay Perimeter · edge DDoS Protection network tier Front Door + WAF OWASP ruleset API Management OIDC · rate limit Control plane · private Registry API mTLS · workload id Policy Service signs bundles Credential Broker no secret transit Execution plane · private, no default egress Runtime Workers per-agent identity Tool Executor scoped token only Session Sandbox Hyper-V · no network Azure Firewall FQDN allow-list Data · restricted Execution Store CMK · private link Definition Store CMK · row security Key Vault HSM 90 day rotation Audit Ledger WORM · 7 years HTTPS 443 signed webhook rejected at edge mTLS + JWT mTLS + run token fenced 15 min token egress inspected private endpoint key logged Security Architecture — Trust Zones and What Crosses Them Person or role External / third party Risk / gap Security / platform Interface / broker Application we own Data store synchronous failure / alternate event / async An agent never inherits its caller's permissions. It carries its own identity, and every zone crossing is authenticated and logged. v 1.0 · owner Security Architecture · date 2026-08 Security Zones and Trust Boundaries Where an attacker arrives, what stops them at each boundary, and what an agent is trusted to do. HTML page SVG draw.io
29 User API Management Microsoft Entra ID Admission Service Runtime Worker Credential Broker Key Vault Downstream Tool 1. sign in, PKCE + MFA 2. id and access token 3. run agent, bearer 4. validate audience and scope 5. forward with caller claims 6. mint agent identity token 7. agent token, narrow scope 8. dispatch with agent token 9. request tool credential 10. check binding and approval 11. fetch by reference 12. 15 minute scoped token 13. invoke with scoped token 14. denied, not in grant Identity and Delegation — From User Sign-In to Tool Credential The user token never leaves the perimeter. An agent gets its own identity, and the intersection of user and agent grant is what it can do. v 1.0 · owner Security Architecture · date 2026-08 Identity and Delegation The path from a user signing in to a tool credential, and what is deliberately not passed along it. HTML page SVG draw.io
30 Layer 1 · Identity and admission Request scoping Tenant Mapping Entra tenant to id Tenant Claim on every token APIM Product per tenant key Request Quota RPS and TPM Layer 2 · Runtime Pooled tier Shared Worker Pool tenant-scoped run Concurrency Cap per tenant Session per Run destroyed after Dedicated tier Dedicated Env own Container Apps Private Endpoints tenant VNet Layer 3 · Data and secrets Stores Cosmos Partition tenant_id key SQL Row Security session context Search Index one per tenant ADLS Container one per tenant Credentials Tenant Key Vault own CMK Rotation Schedule 90 days Layer 4 · Telemetry, cost and audit Segregation Log Scoping tenant dimension ADX Row Policy tenant filter Cost Attribution tag on every call Audit Partition export per tenant Noisy Neighbour pooled tier only Cross-Tenant Read tested each release scopes every query bounds impact denies by default CMK Multi-Tenancy — Four Layers of Isolation Security / platform Interface / broker Decision point Application we own Data store Risk / gap synchronous failure / alternate The pooled tier accepts a bounded noisy-neighbour risk in exchange for cost. The dedicated tier removes it and is priced accordingly. v 1.0 · owner Data & AI Global Practice · date 2026-08 Multi-Tenant Isolation Four layers of isolation, what each one actually prevents, and what the pooled tier still accepts. HTML page SVG draw.io
31 Signal Class Immediate response Containment Residual risk Model provider outage 5xx and timeout Provider Circuit break Fallback route Quality drift Rate limit exhaustion 429 burst Transient Backoff + jitter PTU spillover Latency spike Tool unavailable Health probe red Tool Retry then skip Degraded answer Incomplete result Agent loop divergence Step count rising Agent or model Depth and token cap Terminate run Wasted spend Poison message Delivery count Permanent Dead letter Triage and replay Manual backlog Policy or auth denial 403 on tool call Policy or auth Fail closed Escalate to human Workflow stalls Region loss Probe failure Infrastructure Front Door reroute Replay checkpoints RPO 5 min gap Control plane outage Registry 5xx Infrastructure Admissions stop Cached bundles run No new versions Failure Modes — Signal, Classification, Response and Residual Risk Security / platform Decision point Interface / broker Application we own Risk / gap Queue / topic Person or role Every row ends in a residual risk that is accepted, not eliminated. Those risks are the agenda for the operational review. v 1.0 · owner SRE and Architecture · date 2026-08 Failure Modes and Residual Risk Eight ways this platform fails, what detects each, what contains it, and what is left over. HTML page SVG draw.io
32 Foundation · 6 weeks Landing Zone subscription · Bicep Network and Private Link no public data path Entra and RBAC roles · workload id CI/CD Skeleton build · sign · deploy MVP core · 12 weeks Agent Registry versioned specs Agent Runtime single agent Workflow Engine sequential + branch Model Gateway one provider pool Tool Registry REST + MCP MVP controls · with core Durable State checkpoint · resume Guardrails in · out · tool Execution Tracing OTel GenAI Cost Tracking token accounting API, SDK, Console the whole surface Phase 2 · 2 quarters Multi-Agent Patterns six patterns Human-in-the-Loop approval queues Advanced Memory episodic · semantic Routing and Failover policy driven Evaluation Framework offline + online Multi-Tenancy four layers Phase 3 · beyond Agent Discovery dynamic delegation Autonomous Planning replanning Multi-Region Execution residency routing Policy as Code tenant authored Agent Marketplace internal catalogue prerequisite prerequisite prerequisite Delivery Roadmap — Foundation, MVP and Two Phases Security / platform Application we own Interface / broker Data store synchronous Durable state, tracing and tenancy are prerequisites, not enhancements. Deferring them makes every later phase more expensive. v 1.0 · owner Data & AI Global Practice · date 2026-08 Delivery Roadmap What is built first, what waits, and which items are prerequisites rather than enhancements. HTML page SVG draw.io
Open svg/<view>.svg or drawio/<view>.drawio in draw.io Desktop or at app.diagrams.net to edit. The SVG carries the diagram inside it, so it is both the picture and the source. This folder is self-contained — copy it whole and every link still resolves.
Solution Architecture v1.0 · Data & AI Global Practice · 2026-08 · 32 views · Microsoft Azure32 views · HTML / SVG / draw.io