AI Code Completion Service  ·  View 14 of 21  ·  Runtime

Context Assembly

Spending a token budget, and why the order of what is spent matters more than the amount.

Editable source SVG draw.io All views
Candidate sources Current file prefix + suffix Open tabs up to 20 Recent edits last 10 min Imported symbols Repo snippets retrieved Admission ACL check at request time Ignore rules Poison detector minified, binary Rank Lexical score BM25, in-process Symbol proximity Recency weight Allocate Budget allocator priority order Lowest-priority truncation Order Stable segment cacheable prefix Volatile tail cursor neighbourhood Emit FIM prompt ≤ 8 k tokens Recipe version logged, not the prompt scoped ranked kept tabs only Context Assembly — Spending a Token Budget Application we own Security / platform Decision point synchronous failure / alternate Order is not cosmetic: a volatile segment placed early invalidates the prefix cache on every keystroke and costs 180 ms. v 1.0 · owner Developer Experience Architecture · date 2026-10

The ordering decision

  • The prompt's leading segment must be stable across consecutive keystrokes or the prefix cache is invalidated on every one of them — a volatile segment placed early costs ~180 ms per keystroke.
  • Priority truncation drops the lowest-priority source first and records which sources were dropped, so a quality change is attributable rather than mysterious.
  • Repository permission is checked at request time against the requesting principal, not cached from index build (see view 21).

Assumptions

  • Prompt budget ≤ 8,000 tokens; up to 20 open tabs considered; recent-edit window 10 minutes.
  • Ranking is lexical plus symbol proximity, computed without a round trip the budget cannot afford.

Risks

  • The degradation to open-tabs-only context is invisible to the developer and visible in acceptance rate a day later. It needs its own counter or it becomes a silent quality regression.
◀  Critical Flow — One Keystroke All views Four Fates of One Keystroke  ▶
Solution Architecture v1.0 · Microsoft Azure · Developer Experience Architecture · 2026-10 · 21 views · 15 architecture decision records14-context-assembly-pipeline