Vector Databases
Index families, filtered search, freshness and deletes, sharding, and the operational reality of billion-scale ANN.
11concepts
99flashcards
80minutes of reading
- 01 Choosing an Index Family The four index families and the workload property that selects each, why the recall-latency-memory triangle admits no universal answer, and how to read a published benchmark.
- 02 Coarse Quantisers and the Probe Budget How an inverted-file index turns nearest-neighbour search into a cell-selection problem, why nlist is a square-root decision, and why boundary loss rather than code error sets the recall curve.
- 03 Disk-Based ANN Search and DiskANN How DiskANN serves a billion vectors from one machine with 64 GB of RAM by keeping compressed codes in memory and a Vamana graph on SSD, why the number of disk round trips is the budget that shapes every design choice, and how the Fresh and Filtered variants extend it.
- 04 Filtered Vector Search Why combining a metadata filter with a nearest-neighbour search is harder than either alone, the two naive strategies and how each fails, and what a native implementation does instead.
- 05 Freshness, Deletes and Index Maintenance Why approximate indexes are built for static data, what insertion and deletion do to their structure over time, and the segment architecture that reconciles freshness with query performance.
- 06 Navigable Small World Graphs and HNSW Construction Why greedy search on a nearest-neighbour graph gets stuck, how long-range links and a diversity-based pruning heuristic fix it, and what M, efConstruction and efSearch each actually commit you to.
- 07 Product Quantisation Codebooks and Asymmetric Distance How splitting a vector into subspaces and replacing each with a codebook index turns distance computation into table lookups, why the query is never quantised, and what shrinking the code width buys.
- 08 Sharding and Distributed Vector Search The two ways to partition a vector collection across machines, why one preserves recall and the other preserves latency, and the fan-out arithmetic that decides tail latency.
- 09 Two-Stage Search with Compressed Candidates and Exact Rescoring Why every aggressively compressed index needs a full-precision rerank pass, how k_factor converts a weak ranking requirement into a strong one, and how DiskANN turns an SSD read into the scoring stage.