Approximate Nearest Neighbour Index

Exact nearest-neighbour search compares the query against every vector, which is linear in corpus size and untenable beyond modest scale. ANN indexes return probably the nearest neighbours in sub-linear time.

Recall is the parameter that matters, and it is a dial: higher recall costs latency and memory. A system tuned to 95% recall is silently missing one relevant passage in twenty, which may be fine for general question answering and unacceptable for compliance search. This is a design decision and it is usually left at a default.

The common index families: HNSW — a navigable graph, excellent recall and latency, memory-hungry, and awkward for high-delete workloads. IVF — clustering with probe-count tuning, lower memory, sensitive to data distribution. Product quantisation — compresses vectors to cut memory at some accuracy cost, usually combined with the others.

The operational realities that decide platform choice: filtered search (restricting by tenant, permission or date) is where implementations differ most and where naive approaches either scan too much or lose recall; updates and deletes are more expensive than inserts in graph indexes; and index rebuild is required whenever the embedding model changes, which is a full re-embedding of the corpus.