Retrieval Evaluation
Measuring whether the right context was retrieved, separately from whether the answer was good, because the two failures need different fixes.
The single most useful diagnostic discipline in RAG systems, and the one most often skipped in favour of end-to-end answer scoring.
The two failures look identical to a user and require opposite responses. If the correct passage was never retrieved, no prompt engineering, no larger model and no better instructions will produce a correct answer — the fix is in chunking, embedding, hybrid search or reranking. If the correct passage was retrieved and the answer is still wrong, retrieval is fine and the problem is in the prompt, the model or the context ordering.
Measuring retrieval requires a labelled set: real questions paired with the passages that actually contain the answer. Building it is unglamorous manual work of perhaps a hundred to a few hundred examples, and it is the highest-return investment available in a RAG project.
The metrics: recall at k — is the correct passage in the top k retrieved — which is the one that bounds everything downstream, and precision or mean reciprocal rank for how well it is ranked within that set.
The operational payoff is that changes become measurable rather than argued. Switching embedding models, adjusting chunk size, adding a reranker and enabling hybrid search all become experiments with a number attached, and roughly half of the changes teams are confident about turn out to make things worse.