Search Indexing Service  ·  View 06 of 21  ·  People and journeys

Journey — A Search Engineer Ships a Relevance Change

The trough is promotion: finding out that the change you wanted is a four-hour rebuild.

Editable source SVG draw.io All views
Search Engineer owns relevance Goal — Make "biryani" rank dishes above shop names, provably Trigger — A support thread full of the same complaint Done when — A measured win, live, reversible in a minute 1 · Hypothesise 2 · Measure offline ◆ moment of truth 3 · Promote 4 · Split traffic ◆ moment of truth 5 · Keep or revert What they do Edits boosts Runs judgement set Promotes version Sets 10% split Reads variant metrics What answers Definition registry Evaluation service Swap gate Alias, two variants Engagement log How it feels In control Uneasy Blocked Where it hurts No judgement set yet Analysis change = rebuild What the design owes Offline metric vs live Says which class it is Per-variant metrics Rollback ≤ 60 s Journey — A Search Engineer Ships a Relevance Change The trough is promotion: an index-time change is a four-hour rebuild, and the platform must say so before it is asked for. v 1.0 · owner Data Platform Architecture

What the trough tells the architecture

  • The platform must classify a relevance change as index-time or query-time and say so at authoring time, before anyone schedules a release around the wrong answer.
  • A query-time change rolls back in 60 seconds with no rebuild. An analysis or synonym change is a breaking definition change and takes the full rebuild-and-swap path.
  • The 'no judgement set yet' dip is real and unresolved for a new index: the swap gate has to work without one, which is why document count and sampled diff are separate detectors.

Risks

  • Relevance measured only offline ships changes that win on a judgement set and lose on live traffic; variant metrics are the answer, and they need enough traffic per variant to resolve.
  • A judgement set is a maintained asset. An unmaintained one makes the gate confidently wrong.
◀  Journey — A Merchant Publishes a Change All views Layered Architecture  ▶
Solution Architecture v1.0 · Amazon Web Services · Data Platform Architecture · 2026-1006-journey-ship-a-relevance-change