Search Indexing Service  ·  View 21 of 21  ·  Assurance

Failure Classes and Their Answers

Ten classes, how each is detected, the structural answer, and the residual the design accepts.

Editable source SVG draw.io All views
Detected by Structural answer Accepted residual Source lag or outage Quiet-period alarm Declared degraded freshness Stale answers, honestly labelled Lost change event Sampled reconciliation Targeted re-emit from snapshot ≤ 0.01% divergent Duplicate or reordered Version guard Discard older source version None — model-level Poison document Mapping error Quarantine, partition continues One entity absent until released Schema drift in a source Definition mismatch Quarantine + alarm A field pauses, never drops Enrichment failure Timeout rate Previous value, marked degraded Ranking signal goes stale Bad mapping or relevance Swap gate, variant metrics Refuse swap; rollback by alias 72 h rollback window only Cluster saturation on rebuild Query p99 regression Reserved capacity + throttle ≤ 15% p99 rise accepted Region loss Health checks Read region serves, stale No indexing until log resumes Tenant abuse or hot index Quota + cost ceiling Typed rejection, dedicated move Migration is not instant Failure Classes, Their Answers and the Residual Two classes would change the design: a source that cannot emit a comparable digest, and a tenant whose residency forbids a shared cluster. v 1.0 · owner Data Platform Architecture

Accepted residuals

  • Stale answers during a source outage, labelled as degraded rather than withheld — because absent is worse than stale for everything except suppression.
  • Up to 0.01% of documents divergent at any moment, which is what a sampled reconciliation can actually promise.
  • A 15% query p99 rise for the duration of a rebuild, and a 72-hour window in which rollback is an alias move rather than a rebuild.

What would change the design

  • A source that cannot produce a comparable digest or sequence: lost-document detection then rests entirely on sampling, and the rebuild cadence has to carry the risk instead.
  • A tenant whose residency or key-isolation obligations forbid a shared cluster: placement stops being a performance decision and becomes a compliance one.

Fail-closed / fail-open

  • Fail closed: suppression, moderation, deletion and access filters.
  • Fail open: enrichment values, ranking signals and freshness — degraded and declared, never blocking.
◀  Identity and Access — One Filtered Query All views
Solution Architecture v1.0 · Amazon Web Services · Data Platform Architecture · 2026-1021-failure-classes