Enterprise Generative Search — Azure and Open Source  ·  View 25 of 41  ·  Runtime

Context Assembly

Six slots, their trust levels, their budgets, and what is sacrificed when the window is short.

Editable source SVG draw.io All views
Where it comes from Trust level Token budget When the window is short System policy Versioned prompt registry reviewed like code Trusted the only instructions 700 tokens fixed Never trimmed request fails first Task and output schema Query class template one per class Trusted server-composed 400 tokens fixed Never trimmed schema is the contract Conversation state Last 6 turns, summarised PostgreSQL Semi-trusted the caller wrote it 800 tokens rolling summary Summarise harder oldest turns first Retrieved evidence Evidence set, 8 passages each with an id Untrusted data fenced, never obeyed 6,000 tokens the majority share Drop lowest-ranked passage and say the set was cut Tool results Broker output, typed SQL rows, calculations Untrusted data same fence as evidence 1,200 tokens rows truncated with a count Aggregate, never sample a sample misleads The caller turn This question, verbatim plus resolved scope Trusted as intent not as instruction to tools 300 tokens truncated with notice Never trimmed it is the question Context Assembly — What Occupies the Window, and Under What Rules Compression happens on evidence and history, never on policy or the question. A window that cannot hold the fixed slots is a routing failure, not a prompt problem. v 1.0 · owner Data and AI Global Practice

The decision this view makes

  • Context is assembled from typed slots, not concatenated into a string. The slot is what carries the trust level, and the trust level is what stops evidence being read as instruction.
  • Only two slots are trusted as instructions: the system policy and the task template, both of which are server-composed and versioned. The user's turn is trusted as intent but never as authority over tools or scope.
  • Compression is applied to evidence and history only. A window that cannot hold the fixed slots is a routing failure — the query should have gone to a different class or a different model.

Numbers

  • Explanatory class budget: 700 policy, 400 task, 800 history, 6,000 evidence, 1,200 tool, 300 question — about 9,400 tokens in.
  • Evidence truncation happens on 3.1% of queries and is reported to the user as a trimmed evidence set rather than passed over in silence.

Risks

  • Long-context models make it tempting to skip selection and pass 200 passages. Measured on the golden set, groundedness fell and cost rose fourfold; evidence selection is a quality control, not only a cost one.
  • Tool results are truncated with a count, never sampled. A sampled table read as complete is a wrong answer with no detectable cause.
◀  Hybrid Retrieval and Fusion All views Grounding and Verification  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 41 views · Microsoft Azure and open source25-context-assembly