AI Agent Orchestration Platform  ·  View 25 of 32  ·  5 · Operations

Observability and Tracing

Six signal types across emit, collect, store, consume and act — and the identifier that joins them.

Editable source SVG draw.io All views
Emit Collect Store Consume Act Execution traces OTel GenAI spans OTel Collector Application Insights Execution graph Failure analysis Metrics and SLO Runtime counters Managed Prometheus Monitor workspace Managed Grafana Page on burn rate Logs Structured JSON Container insights Log Analytics KQL incident search Runbook link Token and cost Gateway metric Event Hubs Data Explorer Cost breakdown Budget alert Guardrail decisions Decision event Event Grid Audit ledger Violation report Sentinel rule Quality signals Judge and feedback Evaluation runner Eval results store Agent scorecard Block promotion Observability — Signal against Pipeline Stage Application we own Interface / broker Data store Security / platform Queue / topic Decision point Execution id, workflow id, agent id and correlation id are on every row, so one identifier joins a trace to its cost and its guardrail decisions. v 1.0 · owner SRE · date 2026-08

Decisions

  • OpenTelemetry GenAI semantic conventions are the emission contract, so traces are portable off Azure Monitor if that ever matters
  • Execution id, workflow id, agent id and correlation id are on every signal, so one identifier joins a trace to its cost and its guardrail decisions
  • Prompt and response bodies are captured under privacy policy and stored separately from the trace, referenced by id

What is exposed

  • Real-time execution traces, execution graphs, agent timelines, failure analysis and cost breakdowns
  • Parent and child span relationships across agent, model, tool and guardrail calls
  • Retries, state transitions and guardrail decisions are first-class spans, not log lines

Risks

  • Telemetry volume is a genuine cost line; sampling is head-based for successful runs and always-on for failed and escalated ones
  • A trace containing prompt text is sensitive; the deep link is authorised separately from the trace itself
◀  CI/CD and Environment Promotion All views Cost Management and Attribution  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-08 · 32 views · Microsoft Azure25-observability-and-tracing