Enterprise Generative Search — Azure and Open Source  ·  View 29 of 41  ·  Runtime

Degradation Ladder

Five rungs down from a full answer, and why the bottom rung is still a working product.

Editable source SVG draw.io All views
Healthy All backends up nothing failing Full generative answer claim-bound citations 99.9% of minutes the SLO Rung 1 — model pressure Azure OpenAI throttled 429 or latency spike Route to smaller tier gateway policy Shorter answer, same citations user sees a notice Rung 2 — ranking loss Rerank service down GPU pool unavailable Serve fusion order quality flag recorded Answer with wider evidence confidence lowered Rung 3 — semantic loss Vector arm unavailable index or embed service Lexical-only retrieval recall drops to 0.71 Extractive answer only no synthesis over gaps Rung 4 — index loss Search service degraded region or partition Serve the cache, then fail over paired region alias Cached answers and links staleness stated Rung 5 — floor Generation unavailable safety or policy stop Classic search API always deployable alone Ranked list of documents the product still works search SLO is separate, 99.95% Degradation Ladder — What the User Still Gets When a Dependency Fails Application we own Decision point Risk / gap Journey / task failure / alternate The classic search API is deployable with no model, no GPU and no vector index. That is why the floor of this ladder is a working product rather than an error page. v 1.0 · owner Data and AI Global Practice

The decision

  • Each rung is a defined product behaviour with a user-visible statement, not an unplanned error path. The user is told what changed, because an unexplained shorter answer is read as a worse product rather than a degraded one.
  • The classic search API deploys with no model, no GPU and no vector index. That is why it carries a higher availability target than generative answers and why the floor is a ranked list rather than an error page.
  • Confidence falls as the ladder descends, and the answer says so. Degrading quietly is how a platform loses trust it cannot rebuild.

Numbers

  • Generative answers 99.9%; classic search 99.95%. The gap is the model and GPU dependency, priced deliberately.
  • Lexical-only retrieval measures Recall@50 of 0.78 against 0.92 hybrid — degraded, and still useful.
  • Rungs 1 and 2 are exercised monthly in a game day; rungs 3 to 5 quarterly.

Assumptions

  • The cache can serve a meaningful share of common questions during an index outage, which assumes the 35% hit rate holds under a shifted query mix. It is measured during game days rather than believed.
◀  Multimodal Retrieval All views Deployment Architecture  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 41 views · Microsoft Azure and open source29-degradation-ladder