GraphRAG

Build a knowledge graph from your documents, then traverse it. Answers questions that span many sources, which vector retrieval cannot.

Standard RAG retrieves the passages most similar to your question. That works when the answer sits in a few passages.

It fails on questions where no passage contains the answer:

• What are the main themes across these 200 reports?
• How is this person connected to that company?
• Which of our services depend, transitively, on the payment API?

These need information assembled across sources, not located within one. Vector similarity has no mechanism for that.

The approach

GraphRAG builds an explicit structure at index time instead of only embedding chunks.

Extract entities and relationships. Run a model over each chunk to pull out entities — people, systems, concepts — and the relationships between them. Store as a graph: entities are nodes, relationships are edges.

Resolve duplicates. “Acme Corp”, “Acme”, and “ACME Corporation” must become one node. This step is unglamorous and largely determines whether the graph is useful.

Detect communities. Run a clustering algorithm to find densely connected groups. These correspond to topics.

Summarize each community. Generate a summary of what each cluster is about, then summarize groups of communities, producing a hierarchy from specific to general.

Now two retrieval modes exist that vector search cannot offer.

Local and global queries

Local questions ask about specific entities. Find the relevant nodes, traverse to their neighbours, and gather the connected subgraph. This surfaces information related to your entity even when it never co-occurs with it in any single chunk — which is exactly what multi-hop questions need.

Global questions ask about the corpus as a whole. Answer from the community summaries rather than from chunks. “What are the main themes” is answerable because the themes were computed at index time, not searched for at query time.

That second mode is the genuinely new capability. No amount of chunk retrieval answers a question about the whole corpus, because the answer exists in no chunk.

The cost

This is the honest part, and it is why GraphRAG is not the default.

Indexing is expensive. Entity extraction means a model call per chunk. Community summarization means more. For a large corpus this is orders of magnitude more expensive than embedding, and it is not a one-time cost if documents change.

Extraction quality caps everything. Missed entities and hallucinated relationships propagate into every answer. The graph is only as good as the extraction, and extraction is hallucination-prone.

Entity resolution is genuinely hard. Get it wrong and the graph fragments into duplicates, breaking traversal.

Updates are awkward. Adding a document may require recomputing communities and their summaries.

More moving parts. A graph store alongside a vector store, plus extraction and clustering pipelines.

When it earns the cost

Yes: questions about relationships and connections; corpus-level synthesis; multi-hop reasoning across documents; domains where entities and their links are the subject matter — investigations, org structures, dependency analysis, literature review.

No: factual lookup answerable from one passage; frequently changing corpora; small corpora where a long context window fits everything anyway; anything where standard RAG plus hybrid search plus reranking has not yet been tried.

That last exclusion matters. GraphRAG is sometimes reached for when the actual problem is bad chunking or missing keyword search. Exhaust the cheap improvements first — they solve more real failures.

Lighter alternatives

Several options give some of the benefit at much lower cost:

Metadata graphs. If your documents already carry structured relationships — a code dependency graph, an org chart, a citation network — use it directly. No extraction needed.

Hierarchical summarization without a graph. Summarize documents, then summarize the summaries. Answers global questions without entity extraction.

Agentic retrieval. Let a model issue several searches, following leads across turns. Achieves multi-hop behaviour dynamically rather than by precomputing structure. Higher query-time cost, no indexing cost — often the better trade.

Hybrid. Vector retrieval for specifics, graph or summary retrieval for global questions, routed by query type.

What to remember

• GraphRAG extracts entities and relationships into a graph, then clusters and summarizes it hierarchically.
• Enables local traversal (multi-hop around an entity) and global answers from community summaries.
• The global mode answers corpus-level questions that no single chunk contains — the real new capability.
• Costs are substantial: per-chunk extraction, hard entity resolution, awkward updates, extra infrastructure.
• Try hybrid search, reranking, and better chunking first; consider agentic retrieval as a cheaper route to multi-hop.

Next: Agentic RAG