Repository Context and the Retrieval Problem
Why a model with a large context window still cannot see a codebase, what makes code retrieval different from document retrieval, and the signals that actually locate relevant code.
A production codebase is millions of tokens. Even where a window could hold a meaningful fraction, filling it is expensive, slow, and degrades attention on what matters. The binding problem in AI for software engineering is not generation; it is deciding which few thousand lines the model should see, and code makes that harder than documents do.
Why code retrieval differs
The relevant context is defined by structure, not similarity. To modify a function you need its callers, its callees, the types it uses, its tests, and the conventions of its module. None of those are textually similar to the query, and embedding-based search finds none of them.
Names carry meaning that embeddings dilute. An identifier is a precise handle, and an exact-match search for a symbol is frequently more useful than any semantic search. Lexical retrieval is unusually strong on code for this reason, and hybrid retrieval is not an optimisation but the baseline.
Code is a graph. Imports, calls, inheritance and type references form explicit edges that can be traversed exactly rather than approximated. A parser or a language server gives ground-truth structure that no embedding recovers, and traversal from a seed location is often the highest-precision retrieval available.
Chunking has natural boundaries and they are not fixed size. Splitting mid-function destroys the unit; splitting by file loses granularity. Syntax-aware chunking on function and class boundaries, with the enclosing signatures and imports attached as context, is what preserves interpretability of a fragment.
What works in practice
The effective systems combine several signals rather than relying on one: exact symbol lookup, structural traversal from the edit location, recently viewed or edited files as a proximity signal, and semantic search for the cases where the user described intent rather than named a symbol.
Recency and edit locality deserve emphasis because they are cheap and strong. What the developer has open and has just changed is a better predictor of relevance than any similarity score, and it requires no index.
When it breaks
Indexes go stale on every commit. A codebase changes continuously, so an embedding index is wrong the moment it is built. Incremental update on file change is necessary, and the update path is where most implementations have correctness bugs.
Generated code does not match repository conventions. A model retrieving only the function it edits produces code that works and looks foreign: different error handling, different logging, different naming. Retrieving nearby examples of the pattern rather than only the target is what fixes it, and it costs context.
Cross-repository and dependency context is usually absent. The behaviour that matters often lives in a library outside the repository, and systems that index only the working repository confidently produce code against an API they have never seen.
Monorepos break naive scoping. Retrieving from the whole repository returns irrelevant matches from unrelated services; retrieving from the current directory misses shared code. Ownership and build-graph boundaries are better scoping signals than directory proximity.
12 flashcards for this concept
Click a card to reveal the answer.