Search Evaluation
Pooling and judgments, nDCG and MRR, interleaving, online metrics, and why offline gains vanish online.
4concepts
54flashcards
28minutes of reading
- 01 Interleaving and Online Evaluation Mixing two rankers' results into a single list and attributing clicks gives a within-user paired comparison that detects differences far faster than an A/B test on the same traffic.
- 02 Why Offline Gains Vanish Online The recurring experience that an offline nDCG improvement produces no measurable online effect, and the four distinct mechanisms that cause it.