AI Code Completion Service  ·  View 09 of 21  ·  Structure

Integration Surface

Four APIs, one of which is on a 300 ms budget and three of which may be slow.

Editable source SVG draw.io All views
Callers VS Code plugin reference client JetBrains plugin Neovim plugin Admin console Model promotion CI Code Completion Service Suggestion API streaming, cancellable Telemetry API accept / retain Admin API policy, seats, usage Model API register, split, roll back Dependencies Microsoft Entra ID identity Source hosting repo + ACL AKS GPU pools inference Public-code corpus match index Integration Surface External / third party Security / platform Interface / broker Four APIs, one of which is on a 300 ms budget; the other three are control-plane calls and may be slow. Monitor and the nightly usage export are drawn on views 01 and 18. v 1.0 · owner Developer Experience Architecture · date 2026-10

Decisions

  • The suggestion API is streaming and cancellable; cancellation is a first-class operation with a 50 ms reclamation budget, not a dropped connection.
  • Telemetry is a separate API from suggestion so that a telemetry outage cannot affect a suggestion, and so a tenant can disable the code-bearing stream independently.
  • Model registration and traffic splitting are an API rather than a deployment, which is what makes a 10-minute rollback possible without a client release.

Assumptions

  • Source hosting answers a per-principal repository permission question synchronously, inside the context-assembly budget.
  • The public-code match index is refreshed in batch and is never on the critical path for a non-blocking tenant.

Risks

  • Four plugin surfaces against one contract means version skew is permanent. The version floor and the server-side reduced mode are the only levers, and both are blunt.