Retrieval Grounding

Grounding means the model answers from supplied content rather than from its own priors. It is the mechanism that makes an assistant over a specific corpus useful, and it relocates the engineering problem: output quality becomes dominated by what reaches the model, not by the model.

That relocation is the practically important insight, because it directs effort at retrieval — chunking, hybrid search, reranking, freshness, filtering — rather than at the model layer where teams instinctively look when answers are wrong.

The properties that make grounding real

1. Citations to the retrieved chunks. Without them, a user cannot distinguish a grounded answer from a fabricated one, and the system's central claim is unverifiable. This is the highest-value product affordance in a grounded system.

2. An explicit abstention path. When retrieval confidence is low, "I could not find this" is a far better answer than a fluent invention. Abstention must be designed — a threshold, a prompt structure that permits it, and a product surface that presents it acceptably — because the default behaviour is to answer anyway.

3. Conflict surfaced rather than reconciled. When retrieved sources disagree, silently choosing one produces a confident answer that hides a real disagreement in the corpus.

4. Freshness and authority as ranking signals. In an enterprise corpus, superseded documents typically outnumber current ones and nothing in the text distinguishes them. Faithfully summarising an obsolete document is a grounding success and a product failure.

5. Permission filtering against current truth, failing closed. An index built before a permission change, or filtering applied after ranking, leaks content between users. Grants may be eventually consistent; revocations may not. This is a security property rather than a quality one.

Industry example

Assistants over enterprise documentation exhibit a consistent failure profile: confidently wrong answers whose dominant cause is that retrieval did not surface the right content, followed by outdated content faithfully summarised, followed by chunking that separated a conclusion from its conditions.

Notably, the model is rarely the cause — which is why the effective interventions are hybrid retrieval (pure semantic search misses identifiers, error codes and acronyms, exactly what enterprise questions contain), reranking a small candidate set, structure-preserving chunking with parent context, and explicit archiving of superseded documents.

The same pattern holds for code assistants, product search and support automation: the retrieval layer is the product, and the model is a rendering step.

Failure scenarios

• Grounding claimed without citations, so nothing is verifiable.
• No abstention, so low-confidence retrieval produces fluent invention.
• Retrieval filtered after ranking, returning too few results or leaking content.
• Superseded content ranked equally with current content.
• Chunks without parent context, so retrieved fragments cannot be interpreted or verified.

Trade-offs

Grounding constrains the system to its corpus, which means it cannot answer questions the corpus does not cover — and users will ask those. The abstention path makes this visible, which is honest and is sometimes experienced as the system being less capable.

Citations and abstention also add latency and product complexity, and reranking adds a model call on the critical path. Each is worth it for a system whose value depends on being trusted, and each is skippable for a system where a wrong answer is cheap.

Interview question

"Your assistant confidently answers a question with information from a document that was superseded two years ago. Walk me through every layer where that could have been prevented, and tell me which one you would fix first."