GraphRAG Explained: When Vector Search Isn’t Enough
Your RAG chatbot aces the demo. It answers "What is our refund policy?" instantly. Then someone asks, "Which of our customers are affected if supplier X has a recall?" and it faceplants, because the answer lives across five documents that never share a chunk. Graph RAG was built to close exactly that gap: vector-based retrieval-augmented generation fetches what is similar to your question; GraphRAG fetches what is connected to it.
You'll learn what GraphRAG is, which questions break vector-only RAG, how the knowledge-graph pipeline works (Microsoft, property-graph, lighter alternatives), what it costs, and when you genuinely don't need it.
- What Is GraphRAG?
- Where Vector Search Falls Short: Three Questions That Break Baseline RAG
- How GraphRAG Works: The Architecture From Documents to Knowledge Graph
- Querying GraphRAG: Local, Global, and DRIFT Search
- GraphRAG vs Vector RAG: What Actually Changes
- The GraphRAG Landscape: Microsoft, Neo4j, and Lighter Alternatives
- The Cost Problem: When NOT to Use GraphRAG
- Implementing GraphRAG: A Practical Starter Path
- FAQs
What Is GraphRAG?
GraphRAG (graph-based retrieval-augmented generation) is a RAG architecture that builds a knowledge graph of entities and relationships from your documents, then retrieves connected context by traversing that graph alongside or instead of pulling isolated text chunks by vector similarity enabling multi-hop reasoning and corpus-wide summarization.
The term means two things keep them separate:
- The general pattern any RAG system using a graph structure for retrieval (also written Graph RAG, a "RAG graph," or graph-based RAG).
- Microsoft GraphRAG the open-source Microsoft Research project that popularized the term in 2024, with its own extraction → community → summary pipeline. Unqualified, "GraphRAG" usually means this one.
The pattern entered the mainstream with the Microsoft Research paper "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" (Edge et al., April 2024, arXiv:2404.16130), whose accompanying repository has since crossed 35,000 GitHub stars (35,350, Aug 2026) established architecture, not hype. New to retrieval-augmented generation itself?
Start with how a RAG pipeline works with agents and MCP, then return here for the architecture depth.
Where Vector Search Falls Short: Three Questions That Break Baseline RAG
Vector search is excellent at one job: find chunks whose meaning looks like the query's meaning. Trouble starts when the answer depends not on similarity, but on relationships as in these three question types.
Multi-hop questions: when the answer spans documents
"Which customers are affected if supplier X has a recall?" requires a chain:
supplier X → supplies component Y → Y goes into product Z → Z was sold to customers A, B, C.
Each hop lives in a different document procurement records, bills of materials, sales registers.
Vector search retrieves chunks similar to the wording of your question. "Solen Polymers ships composite resin to Meridian Auto Sources" shares almost no embedding similarity with a question about customer recalls it never uses those words. The chain breaks at hop one; a graph stores each hop as an edge, so the answer is a traversal, not a guess.
Build an AI-First Career, Master the Complete Skillset
Choose from our industry-leading programs designed for career success
Modern Software and AI Engineering Program
Master full-stack development with AI integration
+1000 moreModern Data Science and ML with specialisation in AI
Advanced data science techniques with AI specialization
+1000 moreAdvanced AIML with Specialisation in Agentic AI
Deep dive into AIML with focus on Agentic systems
+1000 moreDevOps, Cloud & AI Platform Engineering
Build and manage AI-powered cloud infrastructure
+1000 moreAI Engineering Advanced Certification by IIT-Roorkee
Premier AI engineering certification from IIT-Roorkee
AI Forward Deployed Engineer Program
Full-stack engineering, production AI and client-facing consulting
+1000 moreGlobal questions: "summarize the main themes" has no matching chunk
"What are the recurring complaints across these 10,000 support tickets?" is aimed at the entire corpus no top-k chunk set contains the answer; it doesn't exist anywhere yet and must be synthesized. This is the exact problem the GraphRAG paper was written to solve: such questions are "inherently a query-focused summarization task, rather than an explicit retrieval task" (Edge et al., 2024), and prior summarization methods don't scale to million-token corpora.
Entity ambiguity and scattered identity
Is "Apple" the company or the fruit? Is "Priya Sharma" in finance's expense report the same "P. Sharma" who signed the legal affidavit? Embeddings blur distinct entities together while treating the same one, spelled differently across documents, as strangers and chunking scatters what is known about each entity across chunk boundaries. A knowledge graph gives each entity exactly one node every mention resolved, every relationship typed so "the same Priya?" is a deterministic graph question.
These are retrieval-architecture problems an agent with ten tools still fetches the same similarity-ranked chunks (see the differences between RAG and agentic AI). In banking and insurance where much of India's IT work happens they also break auditability: vector retrieval can't explain how two facts connect, only that they looked alike.
Worth your time: unsure if your problem is retrieval quality (fixable cheaply) or genuinely relational? Scaler's deep-dive covers the diagnostics:
How GraphRAG Works: The Architecture From Documents to Knowledge Graph
The key insight: an LLM builds the graph for you no manual ontology team required. That single shift separates modern knowledge graph RAG from the hand-built knowledge-graph projects of the 2010s. The pipeline, as implemented in Microsoft's GraphRAG library and official docs, has four stages.
Step 1 Entity and relationship extraction with LLMs
Documents are split into text units (a few hundred tokens each); an LLM reads each and extracts entities (people, organizations, products, concepts) and typed relationships effectively (subject, predicate, object) triples with descriptions and provenance. Input:
"Meridian Auto Sources, a Chennai-based tier-1 supplier, ships brake assemblies to Ashveen Motors. A QC bulletin on 14 May flagged a batch defect in the composite resin Meridian buys from Solen Polymers."
Extracted triples:
| Subject | Predicate | Object |
|---|---|---|
| Meridian Auto Sources | SUPPLIES_TO | Ashveen Motors |
| Meridian Auto Sources | LOCATED_IN | Chennai |
| Meridian Auto Sources | BUYS_FROM | Solen Polymers |
| Solen Polymers | DEFECT_FLAGGED (14 May) | composite resin batch |
The second sentence's dependency Meridian relies on Solen is exactly the cross-document glue vector chunks lose. Extraction quality depends on prompt tuning: Microsoft's docs warn GraphRAG "out of the box may not yield the best possible results" and strongly recommend tuning prompts to your entity types.
Step 2 Building the knowledge graph
Entities become nodes; relationships become edges; duplicate mentions "Meridian Auto Sources," "Meridian Auto," "MAS" merge into one node via entity resolution. Knowledge graphs are classical techniques of knowledge representation applied at LLM scale ideas once encoded by hand, now auto-generated from raw text.
Step 3 Community detection with the Leiden algorithm
Next, the graph is clustered into communities entity groups densely connected internally, loosely connected externally using the Leiden algorithm, an improvement on Louvain that guarantees well-connected communities (Traag et al., arXiv:1810.08473).
Think of communities as topics emerging from the data itself; because Leiden runs hierarchically, you get zoom levels fine clusters ("the Solen Polymers defect cluster") rolling up into broad ones ("supply-chain QC incidents").
Step 4 Community summaries: pre-computed answers to "big" questions
Finally, an LLM writes a summary of every community, bottom-up. These mean the answer to "what are the main themes in this dataset?" is partially pre-computed at index time. That is the fundamental trade of this graph RAG architecture pay heavily once at indexing so global questions become cheap at query time.
How Scaler Transformed Careers in Different Fields
Scaler learners achieved 2.5x salary growth with average post-Scaler CTC reaching ₹23L.
Google
Amazon
Microsoft
Flipkart
Adobe1200+ moreQuerying GraphRAG: Local, Global, and DRIFT Search
The GraphRAG query engine offers three modes, mapping neatly onto the failure modes above:
| Mode | Best for | How it works |
|---|---|---|
| Local search | Entity-centric questions: "Tell me about Solen Polymers and its connections" | Finds relevant entities, fans out to neighbors, relationships, claims, and source text units |
| Global search | Corpus-level questions: "What themes run across all 10,000 tickets?" | Each relevant community drafts a partial answer from its summary; the LLM consolidates them (map-reduce) |
| DRIFT search | Questions needing both detail and context | Dynamic Reasoning and Inference with Flexible Traversal: a "primer" round reads top community reports to draft an answer and spin off follow-ups; local search answers those — local depth guided by global context |
A fourth option, Basic search, is plain top-k vector lookup for questions that never needed a graph. Property graph setups add two mechanics: vector search as an entry point onto the right node, then traversal; and text2cypher, where the LLM writes the Cypher query itself (Neo4j's GraphRAG overview documents them). An LLM choosing which retriever runs per question edges toward agentic RAG.
Stop learning AI in fragments—master a structured AI Engineering Course with hands-on GenAI systems with IIT Roorkee CEC Certification
AI Engineering Course Advanced Certification by IIT-Roorkee CEC
A hands on AI engineering program covering Machine Learning, Generative AI, and LLMs - designed for working professionals & delivered by IIT Roorkee in collaboration with Scaler.
Enrol Now
The GraphRAG Landscape: Microsoft, Neo4j, and Lighter Alternatives
"GraphRAG" now names a family of implementations with very different philosophies and costs:
Microsoft GraphRAG the research-grade pipeline
The official library (MIT-licensed Python, v3.1.1, 35.3k stars as of Aug 2026) implements the full pipeline extraction → Leiden communities → summaries, queried via local, global, and DRIFT search.
Batch-oriented, it shines at corpus sensemaking: big synthetic questions over a stable document set. Its index is flat files (Parquet by default), so no database is mandatory. Trade-offs: expect prompt tuning and the heaviest indexing bill here.
Turn Learning into Career Growth
Neo4j-style property-graph RAG graphs you query directly
The property graph school builds the graph into a real database and queries it directly. The typical Neo4j GraphRAG stack is LangChain's LLMGraphTransformer converting documents into nodes and edges, with retrieval via Cypher, traversals, vector entry points, or text2cypher Neo4j ships a dedicated neo4j-graphrag Python package for it (Neo4j's overview covers the patterns). Its strengths differ: incremental updates are natural (add a document, add its subgraph), structured data fuses with unstructured text, and every answer carries an auditable path. Neutrally noted: Memgraph, FalkorDB, and Amazon Neptune fit the same pattern.
Lighter and cheaper: LightRAG, LazyGraphRAG, and hybrid RAG
Two projects attack the cost problem from opposite ends. LightRAG (from HKU researchers) pairs graph structures with simpler dual-level retrieval (entity-level detail + topic-level abstraction) at far lighter indexing cost; community traction is remarkable the HKUDS/LightRAG repository has ~38,700 stars, more than Microsoft's own. Pilot before betting on it. LazyGraphRAG (Microsoft Research, Nov 2024) defers nearly all LLM work to query time: Microsoft reports indexing cost identical to vector RAG about 0.1% of full GraphRAG's with comparable global-query quality at more than 700× lower query cost (Microsoft Research blog).
Then there's where most production teams land: hybrid RAG vector and graph retrieval fused, a router sending each query to the cheaper adequate path. Choosing here is easier once you understand how LLMs process context our LLM roadmap builds that foundation.
The Cost Problem: When NOT to Use GraphRAG
Here's the part vendor pages skip: full Microsoft-style indexing makes an LLM call for every text unit (extraction, sometimes with multiple "gleaning" passes), plus one per community (summaries at every hierarchy level) before your system answers a single question:
Illustrative corpus: 10,000 support tickets × ~800 tokens ≈ 8M tokens
→ ~13,300 text units (600-token chunks)
→ ~13,300 extraction calls ≈ 13M input + 8M output tokens
→ ~500–1,500 community summary calls ≈ 2–4.5M input + 0.5–1.5M output
─────────────────────────────────────────────────────────────
Indexing bill (Aug 2026 list-price bands — verify your provider's rates):
Mini-class model (~KaTeX parse error: Expected 'EOF', got '–' at position 5: 0.15–̲0.25/M in, \~0.60–1.00/M out): ≈ KaTeX parse error: Expected 'EOF', got '–' at position 2: 5–̲15
Frontier…2.5–5/M in, ~KaTeX parse error: Expected 'EOF', got '–' at position 3: 10–̲15/M out): …100–300
Vector RAG index of the same corpus (embeddings only): ≈ cents
LazyGraphRAG index (0.1% of full GraphRAG, per Microsoft): ≈ pocket change
The meter restarts whenever the corpus changes enough to re-index, and latency follows the same curve global search is measured in seconds and dollars, not milliseconds.
Use GraphRAG when:
- Questions are multi-hop or corpus-level — the three failure modes describe your workload.
- Entities and relationships carry the meaning — people, orgs, transactions, components, policies.
- Explainability is mandatory — compliance, audit trails, regulated domains (BFSI, that's you).
- The corpus is relatively stable — re-indexing weekly, not hourly.
Skip it when:
- Queries are simple lookups — vector search stays faster and cheaper (the GraphRAG docs keep "basic search" for exactly these).
- The corpus changes constantly — graph and summaries go stale together.
- Latency or budget is tight — thousands of indexing calls; heavy global queries.
- You haven't fixed the basics — chunking, retrieval quality, reranking, evaluation in baseline RAG.
That last point trips up more teams than any other: GraphRAG gets deployed when the real problem is bad chunking or a missing reranker fixes an order of magnitude cheaper. A war story: one team pointed Microsoft's pipeline at ~4GB of internal policies and watched frontier-model indexing alone hit five figures in rupees. Worse, logs showed ~80% of questions were simple lookups the graph answered no better than the old vector index.
The fix wasn't abandoning the graph it was a hybrid router: a small classifier sent lookups down the cheap vector path and reserved graph queries for the relational 20%. Spend fell ~6× and answer quality rose; each retriever stayed inside its competence.
The preventive habit: pilot on a 100-document subset measure answer-quality lift versus indexing spend on a fixed question set that deliberately includes simple questions.
Implementing GraphRAG: A Practical Starter Path
Whichever graph RAG implementation the H2-7 framework points you to, keep the first pass small and measurable think of this as a path, not a full GraphRAG tutorial:
- Pick the approach. Microsoft-style for corpus sensemaking; property-graph for connected enterprise data; LightRAG or LazyGraphRAG when indexing budget constrains.
- Stand up a proof-of-concept on a small corpus. Microsoft's route is CLI-driven:
LlamaIndex offers a third path via PropertyGraphIndex, building graph-backed indexes without LangChain.
- Tune extraction prompts to your domain. Default prompts extract generic entities; your corpus has GST numbers, policy codes, SKU hierarchies. Microsoft's docs insist on this step for a reason.
- Evaluate against your existing vector RAG on a fixed question set multi-hop and deliberately simple questions. If the graph doesn't beat the baseline on relational ones, you have a data problem, not an architecture gap.
- Go hybrid for production. Route simple queries to vector search, connected queries to the graph "basic search" exists in the GraphRAG query engine for this reason.
One career note: "vector RAG vs GraphRAG when would you choose which?" is a standard GenAI system-design interview probe now, and the framework above is the expected answer failure modes first, cost math second, hybrid landing zone. For structured practice building RAG end to end, these generative AI courses covering LLMs, RAG, and deployment are good next steps.
FAQs
1. What is GraphRAG?
GraphRAG is a retrieval-augmented generation architecture that extracts entities and relationships from documents into a knowledge graph, then answers questions by traversing those connections and, in Microsoft's implementation, by consulting pre-built community summaries instead of only retrieving text chunks by vector similarity.
2. What is the difference between RAG and GraphRAG?
Standard RAG retrieves chunks semantically similar to the query; GraphRAG retrieves information connected to it following explicit relationships across documents. That enables multi-hop reasoning and corpus-level summarization structurally impossible for vector search, since no single chunk holds a cross-document answer.
3. Is GraphRAG better than RAG?
Only for connected-data questions. It substantially improves multi-hop and "summarize the corpus" answers the original paper reports 72–83% comprehensiveness win rates over naive RAG but costs far more to index and adds real complexity. For simple lookups, vector RAG remains better; many teams run both.
4. How does Microsoft GraphRAG work?
An LLM extracts an entity knowledge graph from your documents; it's clustered into hierarchical communities with the Leiden algorithm; summaries are pre-written per community; queries then run via local search (entity-centric), global search (map-reduce over summaries), or DRIFT search (local search guided by community context).
5. Does GraphRAG require a graph database like Neo4j?
No. Microsoft's GraphRAG stores its index as flat files Parquet by default. Neo4j-style databases suit the property-graph pattern: query the graph directly with Cypher, update incrementally, or fuse it with structured data.
6. When should you not use GraphRAG?
When queries are simple lookups, the corpus changes constantly, latency or budget is tight, or you haven't yet optimized chunking and retrieval quality in baseline RAG. Graph indexing costs thousands of LLM calls before the first query it should never be the first fix.
7. What is the difference between local and global search in GraphRAG?
Local search answers entity-specific questions by exploring an entity's neighbors, relationships, and source text. Global search answers corpus-wide questions ("what are the main themes?") by generating partial answers from community summaries and consolidating them map-reduce over the summaries.
8. Why is GraphRAG expensive?
Because indexing makes LLM calls for every text unit (entity and relationship extraction) plus every community (summaries) thousands of calls before the first query, and again on re-indexing. Lighter variants attack exactly this: LazyGraphRAG defers LLM work to query time (~0.1% of full GraphRAG's indexing cost, per Microsoft).
Conclusion
Vector search answers "what is similar to my question?"; GraphRAG answers "what is connected to my question?" Multi-hop and corpus-level questions need relationships embeddings simply don't store and the price is heavy indexing and operational complexity. Build a solid baseline RAG first, add a graph when your query logs genuinely demand one, and let measured answer quality not hype justify the indexing bill.
Designing retrieval architectures like RAG and GraphRAG and defending those choices in system-design interviews is exactly the kind of applied skill Scaler's AI & Machine Learning program is built around, if you'd like a structured path to master it.
