AI Agent Orchestration Platform  ·  View 07 of 32  ·  2 · Structure

Agent Runtime Internals

What one worker does between leasing a step and committing it, and where the controls sit.

Editable source SVG draw.io All views
Lease Queue Lease session locked Idempotency Check step key Bundle Load pinned version Prepare Context Manager assemble window Memory Read scoped · audited Limit Check tokens · depth · cost Guard in Prompt Shields injection · jailbreak PII Redaction detect · mask Classification Gate residency route Reason Agent Loop framework adapter AI Gateway Call stream · retries Plan or Delegate coordinator hop Act Tool Call Validation schema · permission Tool Executor timeout · retry Session Sandbox code · browser Guard out Structured Output schema enforced Groundedness Check citation bound Escalate to Human on fail Commit Checkpoint durable state Memory Write policy TTL Emit Telemetry OTel GenAI awaits tool result Agent Runtime — Inside One Worker Queue / topic Decision point Application we own Security / platform Interface / broker Data store event / async synchronous The worker holds no long-lived secret and no state between steps. Everything durable is written at Commit. v 1.0 · owner Runtime Engineering · date 2026-08

Decisions

  • The worker is stateless between steps; everything that must survive is written at Commit and nothing is held in memory across a lease
  • Guardrails run in-process on both sides of the model call — a network hop per check would double orchestration overhead
  • The agent framework sits behind an adapter, so a LangGraph or Semantic Kernel agent runs on the same worker image

Limits enforced here

  • Wall-clock timeout, token budget, cost budget and recursion depth, all from the pinned agent spec
  • Tool-call validation happens before invocation, not after: schema, permission grant and argument inspection
  • Structured output is enforced by schema; a non-conforming response is retried once, then escalated

Assumptions

  • Content Safety prompt shields and groundedness detection are available in the execution region
  • A step's work is idempotent given its step key; handlers that cannot be made idempotent must declare a compensation
◀  Integration and Interface Catalogue All views Model Abstraction and Routing  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-08 · 32 views · Microsoft Azure07-agent-runtime-internals