Learning to Rank
Pointwise, pairwise and listwise objectives, LambdaMART, position bias and counterfactual training.
7concepts
133flashcards
52minutes of reading
- 01 LTR Datasets and Ranking Feature Design What the standard learning-to-rank benchmarks (LETOR, MSLR-WEB30K, Yahoo, Istella) actually contain, how their hand-built feature vectors are designed, and what their results do and do not tell you about a production ranker.
- 02 Multi-Stage Ranking Cascades Why production search and recommendation rank in stages of increasing cost over shrinking candidate sets, how to budget candidates and latency per stage, and why the first stage caps what every later stage can achieve.
- 03 Pointwise, Pairwise and Listwise Objectives Ranking is not regression, and the three families of learning-to-rank losses differ in how much of the ranking structure they put inside the objective rather than leaving to a sort.
- 04 Click Models: Cascade, DBN and UBM Generative models of how users scan and click a result page, which let a search log be turned into relevance estimates by modelling what each user probably examined and whether a click satisfied them.
- 05 Counterfactual Learning to Rank Training a ranker on logged clicks while correcting for the bias in how those clicks were generated, which turns a biased log into an unbiased estimate of a ranking objective.
- 06 LambdaRank and LambdaMART The trick of defining a gradient without ever defining a loss, which lets gradient boosting optimise a discontinuous ranking metric directly, and why the result dominated learning to rank for a decade.
- 07 Position Bias and the Examination Hypothesis Clicks measure relevance multiplied by the chance the user looked, so training on raw clicks teaches a ranker to reproduce whatever ranking generated the logs.