Enterprise Generative Search — Azure and Open Source  ·  View 04 of 41  ·  Context and scope

Query Classes and Routing

Six classes of question, and what each is allowed to spend before anything is generated.

Editable source SVG draw.io All views
Recognised by Retrieval plan Generation mode Budget and target Lookup Short, entity-bearing 38% of traffic Hybrid, depth 30 single pass Extractive answer one passage, quoted 4k tokens, 2.5 s P95 cacheable Explanatory How or why phrasing 31% of traffic Hybrid, depth 80 rewrite and expand Synthesis over 6-10 chunks claim-bound 14k tokens, 6 s P95 cacheable Multi-hop or comparative Several entities, contrast 11% of traffic Decomposed sub-queries graph-assisted Agentic loop, 3 hops max contradiction check 45k tokens, 18 s P95 never cached Structured or numeric Aggregation verbs, metrics 9% of traffic NL to SQL on allowlist semantic layer only Table plus narrative figures never restated 8k tokens, 9 s P95 result cached Multimodal Image or audio attached 6% of traffic Cross-modal plus text figure and transcript Answer with a visual crop page-anchored 20k tokens, 12 s P95 never cached Ambiguous or uncovered Low retrieval confidence 5% of traffic Broad recall probe then stop No generation at all clarify or list results Under 1 s, no model cost the cheapest answer Query Classes and the Routing Policy The class is decided before any model is called, so the expensive path is a routing outcome rather than the default. v 1.0 · owner Data and AI Global Practice

Decisions

  • Classification happens before any expensive component runs, so the agentic path is a routing outcome rather than a default. This is the largest single cost control in the design.
  • The ambiguous class does not generate at all. Answering a question the corpus cannot support is worse than saying so, and it is also the cheapest response the platform has.
  • Class thresholds are configuration, not code, and are re-tuned against the golden set rather than by intuition.

Numbers

  • Class mix from six months of intranet search logs plus a pilot: lookup 38%, explanatory 31%, multi-hop 11%, structured 9%, multimodal 6%, ambiguous 5%.
  • Cost per class ranges from 0.004 USD for a cached lookup to 0.180 USD for a full agentic answer — a factor of 45.
  • Classifier accuracy on a 900-query labelled set: 0.93 macro F1, with the ambiguous class deliberately biased towards recall.

Assumptions and risks

  • A mis-classified multi-hop question answered as a lookup produces a confident, partial answer. Verification catches most of these; the abstain rate is watched as the signal that it is not.
  • The class mix will drift as people learn what the platform can do. The routing policy is reviewed monthly against traffic, not set once.
◀  The Evidence Contract All views Actors and Their Core Journeys  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 41 views · Microsoft Azure and open source04-query-classes-and-routing