Classical Ir
4concepts
30flashcards
29minutes of reading
- 01 BM25 and Term Frequency Saturation Why a term appearing twenty times should not score ten times higher than one appearing twice, how BM25 encodes that as a saturating function, and what its two parameters actually control.
- 02 Inverted Indexes and Postings Lists The data structure that makes text search sublinear in corpus size, why postings are stored as sorted document IDs, and how gap encoding turns a list of integers into a few bits each.
- 03 Dynamic Pruning with WAND and Block-Max Retrieving the exact top-k without scoring most of the candidates, by maintaining an upper bound on what each document could score and skipping everything that cannot beat the current threshold.
- 04 Learned Sparse Retrieval Using a language model to assign weights over the vocabulary, including terms not present in the text, so semantic matching runs on an inverted index instead of a vector index.