Grounding Language Models in Structured Knowledge

Vector retrieval finds text that resembles a query. A knowledge graph answers a different kind of question: multi-hop traversals, aggregations, and constraint satisfaction that no similarity search performs. Combining the two is where much of the interesting work in retrieval-augmented systems now sits.

What structure provides

Multi-hop traversal. "Which suppliers of components in products recalled last year also supply our current line" is a graph traversal with a precise answer. Vector search over documents can retrieve passages mentioning some of these things and cannot compose them.

Aggregation and counting. "How many" and "which of these" have exact answers in a graph and approximate ones in a model reading retrieved text, where the count depends on what was retrieved.

Constraint satisfaction. Finding entities satisfying several conditions simultaneously is a query, not a similarity ranking, and a retriever returning the top passages for each condition separately does not intersect them.

Verifiable provenance. A triple has a source, so an answer derived from it is traceable to specific asserted facts rather than to a passage that seemed relevant.

The two patterns

Text-to-query. Translate the natural language question into a formal query, execute it, and present the result. The graph does the reasoning, so multi-hop and aggregation are exact. Everything depends on the translation being correct, and a wrong query returns a confident wrong answer with no signal, since a syntactically valid query over the wrong entities executes perfectly.

Retrieve subgraph, then generate. Identify relevant entities, extract their neighbourhood, serialise it into the prompt, and let the model reason over it. More robust to imprecise questions, since the model can work with approximately right context, and it inherits the model's limitations on counting and multi-hop composition, which is what the graph was supposed to supply.

Systems that work well usually do both: attempt a structured query, fall back to subgraph retrieval when translation fails or the question is not structurally expressible.

When it breaks

Schema linking is the hard part of text-to-query. Mapping "recalled" to the right relation and "last year" to the right temporal predicate requires knowing the schema, and schemas are large, idiosyncratic and often poorly documented. Most text-to-query failures are linking failures rather than syntax failures.

Coverage gaps produce confident absence. An open-world graph returning no results means "not recorded", and a system that reports "there are none" has applied closed-world reasoning to an open-world store. The distinction has to be surfaced to the user rather than resolved silently.

Serialised subgraphs are verbose. Turning a neighbourhood into text consumes context quickly, and a two-hop neighbourhood around a hub can exceed any window. Selecting which relations to include is a retrieval problem inside the retrieval problem.

Graph construction is where the cost sits. Extracting entities and relations from unstructured sources, resolving them, and keeping the result current is a substantial ongoing pipeline. Systems that treat the graph as a static asset discover that its value decays at the rate the world changes, and that maintenance, not construction, is the larger commitment.