GraphRAG and Community Summarisation

Ask a standard RAG pipeline "what are the recurring themes across these 3,000 incident reports?" and watch it fail in a specific, instructive way. The retriever returns the ten chunks most similar to the question, the generator summarises those ten, and the answer describes 0.3% of the corpus with total confidence. Nothing is broken. The query simply is not a retrieval query: no small set of passages contains the answer, because the answer is a property of the whole collection. Microsoft Research named this class global sensemaking and built GraphRAG for it (Edge et al., 2024, From Local to Global: A Graph RAG Approach to Query-Focused Summarization, arXiv:2404.16130).

Indexing: from chunks to a community hierarchy

The insight is to move the expensive work from query time to index time, and to change what gets indexed. GraphRAG runs an LLM over every chunk to extract entities, their descriptions, and the relationships between them, then merges duplicate mentions into a single knowledge graph whose nodes are entities and whose weighted edges count how often two entities co-occur in a described relationship.

That graph is then partitioned with a community detection algorithm (Leiden, in the paper) into a hierarchy: a small number of coarse communities at the top, each recursively split into finer ones. For every community at every level, an LLM writes a summary of what that community is about, working bottom-up so that a parent's summary is built from its children's summaries rather than from raw text. The index is therefore not a pile of vectors; it is a tree of natural-language descriptions of the corpus at several zoom levels.

Global search is map-reduce, not retrieval

At query time a global question skips similarity search entirely. Every community summary at a chosen level is sent to the model in parallel with the question, each producing a partial answer plus a self-assessed helpfulness score. Partials are filtered, ranked, and reduced into one final response.

where \(\mathcal{S}_{\ell}\) is the set of community summaries at hierarchy level \(\ell\). The level is the control knob: root-level summaries are cheap and coarse, leaf-level summaries are expensive and specific. Edge et al. report that on datasets around a million tokens, GraphRAG beat naive RAG on both comprehensiveness and diversity of answers, with low-level community summaries giving the best answers per token and root-level summaries giving competitive answers at a fraction of the query cost.

Local search is the other half of the system and is much closer to conventional RAG: find the entities a question mentions, walk out to their neighbours and the text chunks that mentioned them, and answer from that neighbourhood. Most production deployments need both, with a router deciding which mode a question belongs to.

The distinction that matters: global versus local

Confusing the two is the most common failure. "Who signed the Q3 vendor agreement?" is local; the answer sits in one place and vector search will find it faster and cheaper than any graph traversal. "How has our vendor risk posture changed?" is global; no chunk contains it. Running global search on a local question burns money on hundreds of map calls to answer something one retrieval would have handled. Running local search on a global question produces the confident 0.3% answer described above.

When it breaks

• Indexing cost is the whole story. Extraction runs an LLM over every chunk, and summarisation runs it again over every community at every level. For a large corpus this is orders of magnitude more expensive than embedding, and it is paid before the first question is asked. If your corpus is small, or your questions are all local, this cost has no return.
• The graph inherits extraction errors. Entity resolution is the hard part: Acme Corp, ACME, and Acme Corporation become three nodes unless merged, splitting a community and diluting its summary. Errors compound upward, because a wrong child summary is faithfully incorporated into its parent.
• Updates are not incremental by nature. Adding documents can change community structure, which invalidates summaries above the changed region. Implementations offer incremental modes, but a corpus with high churn pays repeatedly.
• Summaries are lossy by construction, and that is the point. A community summary is a compression of thousands of tokens into hundreds; anything the summariser judged unimportant is unavailable at query time no matter how the question is phrased. Global search cannot retrieve a detail it summarised away.
• Evaluation is genuinely hard. There is no gold answer to "what are the main themes", which is why the paper evaluates with an LLM judge on comprehensiveness, diversity, and empowerment rather than with recall against labelled passages (see rag evaluation and groundedness). Treat reported win rates as directional.