Inference & Serving
30 min
What a Vector Index Actually Costs: The Router, the Quantiser, and the Recall Ceiling
In the NeurIPS'21 billion-scale ANN challenge, entrants were ranked on recall at a fixed 10,000 queries per second. The best standard-hardware submissions landed around 0.71 to 0.79 recall@10, not 0.99. Every vector index is two …