From Local to Global: A Graph RAG Approach to Query-Focused Summarization

Darren EdgeHa TrinhNewman ChengJoshua BradleyA. ChaoApurva N. ModySteven TruittJonathan Larson

article2024arXiv2,164 citations

Introduces GraphRAG, a framework that combines knowledge graph extraction and hierarchical community summarization to answer corpus-level sensemaking questions that defeat standard retrieval-augmented generation.

Listen

Modern organizations increasingly rely on large language models to answer questions using private or domain-specific document collections. While conventional retrieval systems excel at fetching isolated facts from a text corpus, they fail when users ask broad, high-level sensemaking questions such as identifying major themes or summarizing global trends. At the same time, traditional text summarization techniques do not scale effectively to the large volumes of data indexed by modern systems.

The article introduces GraphRAG, a graph-based framework designed to perform global query-focused summarization over large, private text collections. The authors set out to demonstrate that building a hierarchical graph index enables language models to answer broad sensemaking queries with greater comprehensiveness and diversity than conventional retrieval-augmented generation methods.

To achieve this, the approach first uses a language model to extract key entities, relationships, and claims across text chunks to build a knowledge graph. It then applies community detection algorithms to group related entities hierarchically and generates pre-computed summaries for each community. When a user submits a global query, the system generates intermediate answers from these community summaries in parallel and synthesizes them into a final global response. The authors evaluated this method against conventional semantic search and direct text summarization across two representative 1-million-token datasets (podcast transcripts and news articles) using both automated language model evaluators and factual claim extraction metrics.

The article reports several key findings. First, GraphRAG substantially outperformed conventional retrieval methods, achieving win rates between 72% and 83% for answer comprehensiveness and 62% to 82% for answer diversity. Second, validation through factual claim extraction confirmed that GraphRAG generated significantly more verifiable factual claims per answer (averaging 31 to 34 claims) compared to conventional retrieval (averaging 25 to 26 claims). Third, using high-level root community summaries reduced token consumption by 97% compared to whole-text summarization while still retaining significant advantages over conventional retrieval. Finally, while conventional retrieval produced more direct and concise answers, GraphRAG provided far greater depth and breadth across disparate topics.

These findings indicate that integrating hierarchical knowledge graphs into retrieval architectures resolves a major blind spot in generative AI. By structuring unstructured text into modular community summaries, organizations can perform corpus-wide intelligence analysis, thematic discovery, and high-level synthesis without suffering from context window limits or the prohibitive token costs of brute-force summarization.

Decision-makers implementing generative AI over large document repositories should consider deploying graph-based indexing when user workflows require high-level summaries and broad discovery. For cost-sensitive applications with iterative querying, the authors suggest utilizing root-level community summaries to maintain high performance at a fraction of the operational token cost. Future development should explore hybrid systems that combine local semantic search with hierarchical community drill-down capabilities.

Readers should note that the evaluation was conducted primarily on two datasets in the 1-million-token range using GPT-4, meaning performance may vary across other domains or smaller language models. Furthermore, because generated summaries synthesize abstract themes across multiple documents, organizations should maintain clear AI disclosure policies and verify critical claims to mitigate the risk of model fabrication.

Microsoft graph-based graphrag language model query-focused summarization Graph RAG knowledge graph

arXiv: 2404.16130 microsoft/graphrag

Paper: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Patrick Lewis et al. (2020). This seminal paper introduces the retrieval-augmented generation (RAG) framework that GraphRAG directly extends and benchmarks against for question answering.
Paper: Retrieval-Augmented Generation for Large Language Models: A Survey, Yunfan Gao et al. (2023). This survey provides a comprehensive taxonomy of standard and advanced RAG architectures, contextualizing the limitations GraphRAG aims to solve.
Paper: A Survey on Knowledge Graphs: Representation, Acquisition, and Applications, Shaoxiong Ji et al. (2020). This survey details knowledge graph representations and entity-relation extraction methods foundational to GraphRAG's graph-indexing pipeline.
Paper: G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment, Yang Liu et al. (2023). This work establishes LLM-based evaluation protocols and scoring criteria that inform GraphRAG's assessment of comprehensiveness and diversity.
Paper: Lost in the Middle: How Language Models Use Long Contexts, Nelson F. Liu et al. (2024). This study demonstrates how large language models struggle to utilize information located in the middle of long contexts, motivating GraphRAG's structured summarization approach.

Paper: Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents, Theo Rusu et al. (2026). This research investigates graph-structured memory versus flat RAG in long-term LLM agent contexts, extending the practical application and maintenance of graph indexes.
Paper: Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents, Shuo Ji et al. (2026). This paper expands graph-augmented retrieval into dynamic multi-step agent reasoning through interactive graph traversal.
Paper: On the Theoretical Limitations of Embedding-Based Retrieval, Orion Weller et al. (2026). This work establishes theoretical bounds on dense vector retrieval capacity, explaining mathematically why modular and graph-based indexing approaches become necessary at corpus scale.
Paper: Idea2Story: An Automated Pipeline for Transforming Research Concepts into Complete Scientific Narratives, Tengyue Xu et al. (2026). This paper applies offline knowledge-graph construction and structured narrative retrieval to the scientific discovery domain.

Cover for From Local to Global: A Graph RAG Approach to Query-Focused Summarization

Abstract

The use of retrieval-augmented generation (RAG) to retrieve relevant information from an external knowledge source enables large language models (LLMs) to answer questions over private and/or previously unseen document collections. However, RAG fails on global questions directed at an entire text corpus, such as "What are the main themes in the dataset?", since this is inherently a query-focused summarization (QFS) task, rather than an explicit retrieval task. Prior QFS methods, meanwhile, do not scale to the quantities of text indexed by typical RAG systems. To combine the strengths of these contrasting methods, we propose GraphRAG, a graph-based approach to question answering over private text corpora that scales with both the generality of user questions and the quantity of source text. Our approach uses an LLM to build a graph index in two stages: first, to derive an entity knowledge graph from the source documents, then to pregenerate community summaries for all groups of closely related entities. Given a question, each community summary is used to generate a partial response, before all partial responses are again summarized in a final response to the user. For a class of global sensemaking questions over datasets in the 1 million token range, we show that GraphRAG leads to substantial improvements over a conventional RAG baseline for both the comprehensiveness and diversity of generated answers.

Topics

graph-based graphrag language model question answering retrieval-augmented generation sense making text document electronic document user information

Keywords

query-focused summarization Graph RAG knowledge graph community detection community hierarchy element summaries semantic search graph index

1 Introduction
2 Background
2.1 RAG Approaches and Systems
2.2 Using Knowledge Graphs with LLMs and RAG
2.3 Adaptive benchmarking for RAG Evaluation
2.4 RAG evaluation criteria
3 Methods
3.1 GraphRAG Workflow
3.1.1 Source Documents →\rightarrow Text Chunks
3.1.2 Text Chunks →\rightarrow Entities & Relationships
3.1.3 Entities & Relationships →\rightarrow Knowledge Graph
3.1.4 Knowledge Graph →\rightarrow Graph Communities
3.1.5 Graph Communities →\rightarrow Community Summaries
3.1.6 Community Summaries →\rightarrow Community Answers →\rightarrow Global Answer
3.2 Global Sensemaking Question Generation
3.3 Criteria for Evaluating Global Sensemaking
4 Analysis
4.1 Experiment 1
4.1.1 Datasets
4.1.2 Conditions
4.1.3 Configuration
4.2 Experiment 2
5 Results
5.1 Experiment 1
5.2 Experiment 2
6 Discussion
6.1 Limitations of evaluation approach
6.2 Future work
7 Conclusion
References
A Entity and Relationship Extraction Approach
A.1 Entity Extraction
A.2 Self-Reflection
B Example Community Detection
C Context Window Selection
D Example Answer Comparison
E System Prompts
E.1 Element Instance Generation
E.2 Community Summary Generation
E.3 Community Answer Generation
E.4 Global Answer Generation
F Evaluation Prompts
F.1 Relative Assessment Prompt
F.2 Relative Assessment Metrics
G Statistical Analysis

Knowls

Knowl 1 — GraphRAG Knowledge Graph Indexing Pipeline
model/method
The GraphRAG indexing pipeline transforms an unindexed text corpus into a hierarchical, community-structured knowledge graph index through four sequential stages:
1. Text Chunking: Source documents are divided into fixed-size text chunks (such as 600 tokens with 100-token overlaps).
2. Element Extraction: A large language model (LLM) extracts entity instances (capitalized name, type, and description), relationship instances (source entity, target entity, relationship description, and numeric strength score), and factual claims (verifiable statements with dates, events, or interactions).
3. Knowledge Graph Assembly: Multiple mentions of entities and relationships across chunks are merged using exact string matching for entity reconciliation. Entity descriptions are aggregated and summarized into single node representations. Duplicate relationships between the same entity pairs are merged into weighted edges where edge weight equals occurrence count. Claims are linked to their corresponding entity nodes.
4. Hierarchical Community Partitioning: Leiden community detection is applied recursively to partition the graph into nested, modular sub-communities of strongly connected entities, creating a multi-level community hierarchy (from root level $C_0$ down to leaf communities $C_3$ ) where every level provides a mutually exclusive, collectively exhaustive partition of the graph's nodes.
Knowl 2 — Hierarchical Community Summarization Algorithm
algorithm
GraphRAG generates structured, report-like summaries for all communities across the graph hierarchy using a bottom-up recursive procedure that respects the LLM context window limit $L$ .
Input: Graph hierarchy with levels $C_0, C_1, \dots, C_D$ , community node/edge/claim descriptions, context token limit $L$
Output: Community summary reports for all communities across all levels
for level $d = D$ down to 0 do
    for each community $c \in C_d$ do
        if $c$ is a leaf-level community then
            Rank internal edges in descending order of combined node degree: $(deg(u) + deg(v))$
            Initialize prompt context $P \leftarrow \emptyset$
            for each edge $(u, v)$ in ranked order do
                 $E \leftarrow \{ \text{description}(u), \text{description}(v), \text{description}(u, v), \text{claims}(u, v) \}$
                if $\text{TokenCount}(P \cup E) \le L$ then
                     $P \leftarrow P \cup E$
                else
                    break
            Generate summary report for $c$ using LLM given context $P$
        else
            if all raw element summaries of $c$ fit within $L$ then
                Generate summary report for $c$ from all raw element summaries
            else
                Rank sub-communities of $c$ in descending order of constituent token length
                Initialize community representation with raw element summaries
                for each sub-community $s \subseteq c$ in ranked order do
                    Substitute raw element summaries of $s$ with the pre-generated summary of $s$
                    if total tokens $\le L$ then
                        break
                Generate summary report for $c$ using LLM given substituted context
return all generated community summary reports
Knowl 3 — Map-Reduce Query-Focused Summarization for Global Sensemaking
model/method
GraphRAG answers global sensemaking queries over an entire corpus through a multi-stage map-reduce workflow operating on pre-generated community summaries at a selected hierarchy level:
1. Context Preparation: All community summaries at the chosen hierarchical level (e.g., $C_0, C_1, C_2,$ or $C_3$ ) are randomly shuffled and partitioned into context chunks of a pre-specified token size to distribute semantically related information across chunks rather than concentrating it in a single window.
2. Map Phase (Intermediate Community Answers): For each summary chunk in parallel, the LLM generates an intermediate answer to the user's query along with an integer helpfulness rating $\in [0, 100]$ reflecting how relevant and useful that chunk's summary was for addressing the query. Intermediate responses with a helpfulness score of 0 are filtered out.
3. Reduce Phase (Global Answer Synthesis): Remaining intermediate community answers are sorted in descending order of their helpfulness scores and greedily added to a new context window until the LLM context token limit is reached. The LLM then synthesizes this aggregated context into a single, comprehensive global answer with explicit citations to data reports.
Knowl 4 — Self-Reflection Gleaning for Entity Extraction
model/method
When extracting entities from text chunks, larger chunk sizes reduce total LLM invocations but degrade extraction recall (for instance, extracting nearly half as many entity mentions at 2400 tokens compared to 600 tokens). To maintain high recall with larger chunk sizes, GraphRAG employs an iterative self-reflection "gleaning" procedure:
1. The LLM performs an initial extraction of entities and relationships from the text chunk.
2. The extracted element list is fed back to the LLM with a prompt asking whether any entities were missed, forcing a boolean determination using an extreme logit bias of $+100$ on yes/no completion tokens.
3. If the model indicates unextracted entities remain, a continuation prompt stating that many entities were missed is issued, directing the model to extract the missing entities.
4. Steps 2–3 repeat for a predetermined maximum number of self-reflection iterations, increasing detected entity references without introducing spurious noise.
Knowl 5 — Adaptive Benchmarking for Sensemaking Question Generation
algorithm
To evaluate global sensemaking systems without ground-truth answers or low-level fact extraction bias, GraphRAG generates corpus-level evaluation queries dynamically from high-level corpus descriptions:
Input: High-level corpus description $D$ , user persona count $K$ , task count per persona $N$ , question count per task $M$
Output: Benchmark set of $K \times N \times M$ global sensemaking questions
Prompt LLM with $D$ to generate $K$ distinct user personas $P = \{p_1, p_2, \dots, p_K\}$
Initialize question benchmark set $Q \leftarrow \emptyset$
for each persona $p \in P$ do
    Prompt LLM with $(D, p)$ to identify $N$ realistic user tasks $T_p = \{t_1, t_2, \dots, t_N\}$
    for each task $t \in T_p$ do
        Prompt LLM with $(D, p, t)$ to generate $M$ high-level sensemaking questions
        Ensure each question requires global corpus synthesis and cannot be resolved by isolated fact retrieval
         $Q \leftarrow Q \cup \{\text{generated questions for } (p, t)\}$
return $Q$
Knowl 6 — Claim-Based Evaluation Framework for Summarization Quality
model/method
To validate LLM-as-a-judge comparative ratings without ground truth, GraphRAG evaluates answer quality using extracted atomic factual claims:
- Atomic Claim Extraction: An LLM-based claim extractor parses generated responses into atomic, self-contained factual claims (defined as statements explicitly asserting verifiable facts), with duplicate claims within the same response eliminated.
- Comprehensiveness Metric: The total count of unique factual claims contained in the generated answer.
- Diversity Metric: Agglomerative clustering with complete linkage is applied to the extracted claims of an answer using the distance function: $d(c_i, c_j) = 1 - \text{ROUGE-L}(c_i, c_j)$ Diversity is defined as the number of distinct claim clusters formed at a specified distance threshold $\tau \in [0.5, 0.8]$ .
Knowl 7 — Comparative Sensemaking Performance of GraphRAG vs Vector RAG
empirical result
Across 125 global sensemaking questions on two ~1M-token corpora (Podcast Transcripts and News Articles) evaluated using GPT-4 head-to-head judgments across 5 independent replicates:
- Comprehensiveness: All global methods (GraphRAG levels $C_0, C_1, C_2, C_3$ and direct text summarization $TS$ ) significantly outperformed Vector RAG ( $SS$ ), achieving win rates of 72%–83% on Podcast Transcripts ( $p < 0.001$ ) and 72%–80% on News Articles ( $p < 0.001$ ).
- Diversity: Global methods achieved win rates of 75%–82% on Podcasts ( $p < 0.001$ ) and 62%–71% on News ( $p < 0.01$ ) over Vector RAG.
- Directness (Control): Vector RAG consistently won on directness (54%–65% win rates against global methods), confirming that vector RAG provides more concise answers at the expense of breadth and multi-faceted coverage.
- Graph Community Summaries vs. Source Text Summarization: Intermediate and low-level GraphRAG summaries ( $C_2, C_3$ ) provided modest but statistically significant gains over direct text summarization ( $TS$ ) in comprehensiveness (win rates of 57% on Podcasts and 64% on News, $p < 0.001$ ) and diversity (57% on Podcasts, $p = 0.036$ ; 60% on News, $p < 0.001$ ).

Knowl 8 — Context Token Efficiency Across Graph Community Hierarchy Levels

data/table

GraphRAG hierarchical community levels provide substantial query-time context token savings compared to map-reduce over raw text chunks ( $TS$ ). The table summarizes context units, token counts, and token cost relative to $TS$ for Podcast Transcripts (~1.0M tokens) and News Articles (~1.7M tokens):

	Podcast Transcripts					News Articles
Metric	C0	C1	C2	C3	TS	C0	C1	C2	C3	TS
Units	34	367	969	1310	1669	55	555	1797	2142	3197
Tokens	26,657	225,756	565,720	746,100	1,014,611	39,770	352,641	980,898	1,140,266	1,707,694
% Max	2.6%	22.2%	55.8%	73.5%	100.0%	2.3%	20.7%	57.4%	66.8%	100.0%

Root-level community summaries ( $C_0$ ) require 97.4% to 97.7% fewer context tokens per query than raw text map-reduce ( $TS$ ), while preserving a 72% win rate over Vector RAG in comprehensiveness and 62%–77% in diversity.

Knowl 9 — Empirical Claim Extraction and Clustering Validation

data/table

Validation using automated atomic claim extraction and complete-linkage claim clustering ( $1 - \text{ROUGE-L}$ ) confirms that global GraphRAG methods produce higher factual claim density and conceptual diversity than vector RAG ( $SS$ ):

Metric / Dataset	C0	C1	C2	C3	TS	SS
Avg. Claims (News)	34.18	32.50	31.62	33.14	32.89	25.23
Avg. Claims (Podcast)	32.21	32.20	32.46	32.28	31.39	26.50
News Clusters ( $\tau=0.5$ )	23.42	21.85	21.90	22.13	21.80	17.92
News Clusters ( $\tau=0.6$ )	21.65	20.38	20.30	20.52	20.13	16.78
News Clusters ( $\tau=0.7$ )	20.19	19.06	19.03	19.13	18.62	15.80
News Clusters ( $\tau=0.8$ )	18.86	17.78	17.82	17.79	17.30	14.80
Podcast Clusters ( $\tau=0.5$ )	23.16	22.62	22.52	21.93	21.14	18.55
Podcast Clusters ( $\tau=0.6$ )	21.65	21.33	21.21	20.62	19.70	17.39
Podcast Clusters ( $\tau=0.7$ )	20.41	20.04	19.79	19.22	18.08	16.28
Podcast Clusters ( $\tau=0.8$ )	19.26	18.77	18.46	17.89	16.66	15.07

All global search conditions ( $C_0$ -- $C_3$ ) and text summarization ( $TS$ ) extract significantly more claims than vector RAG ( $SS$ ) ( $p < 0.05$ ). Non-tied LLM comparative preferences aligned with the claim-based metrics in 78% of pairwise comparisons for comprehensiveness and 69%–70% for diversity.

Knowl 10 — Context Window Size Degradation in Retrieval-Augmented Generation
empirical result
In evaluations comparing LLM context window limits of 8k, 16k, 32k, and 64k tokens for query answering with gpt-4-turbo:
- The smallest context window (8k tokens) achieved higher performance on answer comprehensiveness, winning an average of 58.1% of pairwise comparisons against larger context windows.
- Answers generated with an 8k window performed comparably to larger context windows on diversity (52.4% win rate) and empowerment (51.3% win rate).
- This demonstrates that simply enlarging the context window of a flat vector RAG system does not resolve global sensemaking queries due to attention degradation over long sequences ("lost in the middle"), justifying fixed 8k-token chunking budgets during map-reduce processing.

Coverage note — No substantial contributed material was omitted; full raw prompt templates and qualitative output examples from the appendices were synthesized into the corresponding method, algorithm, and empirical evaluation knowls.

References

1.Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
2.Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., et al. (2023). Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
3.Baek, J., Aji, A. F., and Saffari, A. (2023). Knowledge-augmented language model prompting for zero-shot knowledge graph question answering. arXiv preprint arXiv:2306.04136.
4.Ban, T., Chen, L., Wang, X., and Chen, H. (2023). From query tools to causal architects: Harnessing large language models for advanced causal discovery from data.
5.Barlaug, N. and Gulla, J. A. (2021). Neural networks for entity matching: A survey. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(3):1–37.
6.Baumel, T., Eyal, M., and Elhadad, M. (2018). Query focused abstractive summarization: Incorporating query relevance, multi-document coverage, and summary length constraints into seq2seq models. arXiv preprint arXiv:1801.07704.
7.Blondel, V. D., Guillaume, J.-L., Lambiotte, R., and Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment, 2008(10):P10008.
8.Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
9.Cheng, X., Luo, D., Chen, X., Liu, L., Zhao, D., and Yan, R. (2024). Lift yourself up: Retrieval-augmented text generation with self-memory. Advances in Neural Information Processing Systems, 36.
10.Christen, P. and Christen, P. (2012). The data matching process. Springer.
11.Chung, J., Pedigo, B. D., Bridgeford, E. W., Varjavand, B. K., Helm, H. S., and Vogelstein, J. T. (2019). Graspy: Graph statistics in python. Journal of Machine Learning Research, 20(158):1–7.
12.Dang, H. T. (2006). Duc 2005: Evaluation of question-focused summarization systems. In Proceedings of the Workshop on Task-Focused Summarization and Question Answering, pages 48–55.
13.Elmagarmid, A. K., Ipeirotis, P. G., and Verykios, V. S. (2006). Duplicate record detection: A survey. IEEE Transactions on knowledge and data engineering, 19(1):1–16.
14.Es, S., James, J., Espinosa-Anke, L., and Schockaert, S. (2023). Ragas: Automated evaluation of retrieval augmented generation. arXiv preprint arXiv:2309.15217.
15.Etzioni, O., Cafarella, M., Downey, D., Kok, S., Popescu, A.-M., Shaked, T., Soderland, S., Weld, D. S., and Yates, A. (2004). Web-scale information extraction in knowitall: (preliminary results). In Proceedings of the 13th International Conference on World Wide Web, WWW ’04, page 100–110, New York, NY, USA. Association for Computing Machinery.
16.Feng, Z., Feng, X., Zhao, D., Yang, M., and Qin, B. (2023). Retrieval-generation synergy augmented large language models. arXiv preprint arXiv:2310.05149.
17.Fortunato, S. (2010). Community detection in graphs. Physics reports, 486(3-5):75–174.
18.Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., and Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.
19.He, X., Tian, Y., Sun, Y., Chawla, N. V., Laurent, T., LeCun, Y., Bresson, X., and Hooi, B. (2024). G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. arXiv preprint arXiv:2402.07630.
20.Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., and Zhou, D. (2023). Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798.
21.Jacomy, M., Venturini, T., Heymann, S., and Bastian, M. (2014). Forceatlas2, a continuous graph layout algorithm for handy network visualization designed for the gephi software. PLoS ONE 9(6): e98679. https://doi.org/10.1371/journal.pone.0098679.
22.Jin, D., Yu, Z., Jiao, P., Pan, S., He, D., Wu, J., Philip, S. Y., and Zhang, W. (2021). A survey of community detection approaches: From statistical modeling to deep learning. IEEE Transactions on Knowledge and Data Engineering, 35(2):1149–1170.
23.Kang, M., Kwak, J. M., Baek, J., and Hwang, S. J. (2023). Knowledge graph-augmented language models for knowledge-grounded dialogue generation. arXiv preprint arXiv:2305.18846.
24.Khattab, O., Santhanam, K., Li, X. L., Hall, D., Liang, P., Potts, C., and Zaharia, M. (2022). Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive nlp. arXiv preprint arXiv:2212.14024.
25.Kim, D., Xie, L., and Ong, C. S. (2016). Probabilistic knowledge graph construction: Compositional and incremental approaches. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM ’16, page 2257–2262, New York, NY, USA. Association for Computing Machinery.
26.Kim, G., Kim, S., Jeon, B., Park, J., and Kang, J. (2023). Tree of clarifications: Answering ambiguous questions with retrieval-augmented large language models. arXiv preprint arXiv:2310.14696.
27.Klein, G., Moon, B., and Hoffman, R. R. (2006). Making sense of sensemaking 1: Alternative perspectives. IEEE intelligent systems, 21(4):70–73.
28.Kosinski, M. (2024). Evaluating large language models in theory of mind tasks. Proceedings of the National Academy of Sciences, 121(45):e2405460121.
29.Kuratov, Y., Bulatov, A., Anokhin, P., Sorokin, D., Sorokin, A., and Burtsev, M. (2024). In search of needles in a 11m haystack: Recurrent memory finds what llms miss.
30.LangChain (2024). Langchain graphs. https://langchain-graphrag.readthedocs.io/en/latest/.
31.Laskar, M. T. R., Hoque, E., and Huang, J. (2020). Query focused abstractive summarization via incorporating query relevance and transfer learning with transformer models. In Advances in Artificial Intelligence: 33rd Canadian Conference on Artificial Intelligence, Canadian AI 2020, Ottawa, ON, Canada, May 13–15, 2020, Proceedings 33, pages 342–348. Springer.
32.Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Kuttler, H., Lewis, M., Yih, W.-t., Rockt¨aschel, T., et al. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.
33.Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. (2023). Lost in the middle: How language models use long contexts. arXiv:2307.03172.
34.LlamaIndex (2024). GraphRAG Implementation with LlamaIndex - V2. https://github.com/run-llama/llama_index/blob/main/docs/docs/examples/cookbooks/GraphRAG_v2.ipynb.
35.Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2024). Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36.
36.Manakul, P., Liusie, A., and Gales, M. J. (2023). Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896.
37.Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., and Chen, W. (2020). Generation-augmented retrieval for open-domain question answering. arXiv preprint arXiv:2009.08553.
38.Martin, S., Brown, W. M., Klavans, R., and Boyack, K. (2011). Openord: An open-source toolbox for large graph layout. SPIE Conference on Visualization and Data Analysis (VDA).
39.Melnyk, I., Dognin, P., and Das, P. (2022). Knowledge graph generation from text.
40.Metropolitansky, D. and Larson, J. (2025). Towards effective extraction and evaluation of factual claims.
41.Microsoft (2023). The impact of large language models on scientific discovery: a preliminary study using gpt-4.
42.Mooney, R. J. and Bunescu, R. (2005). Mining knowledge from text using information extraction. SIGKDD Explor. Newsl., 7(1):3–10.
43.NebulaGraph (2024). Nebulagraph launches industry-first graph rag: Retrieval-augmented generation with llm based on knowledge graphs. https://www.nebula-graph.io/posts/graph-RAG.
44.Neo4J (2024). Get started with graphrag: Neo4j’s ecosystem tools. https://neo4j.com/developer-blog/graphrag-ecosystem-tools/.
45.Newman, M. E. (2006). Modularity and community structure in networks. Proceedings of the national academy of sciences, 103(23):8577–8582.
46.Ni, J., Shi, M., Stammbach, D., Sachan, M., Ash, E., and Leippold, M. (2024). AFaCTA: Assisting the annotation of factual claim detection with reliable LLM annotators. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1890–1912, Bangkok, Thailand. Association for Computational Linguistics.
47.OpenAI (2023). Chatgpt: Gpt-4 language model.
48.Padmakumar, V. and He, H. (2024). Does writing with language models reduce content diversity? ICLR.
49.Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830.
50.Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., and Shoham, Y. (2023). In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316–1331.
51.Ranade, P. and Joshi, A. (2023). Fabula: Intelligence report generation using retrieval-augmented narrative construction. arXiv preprint arXiv:2310.13848.
52.Salminen, J., Liu, C., Pian, W., Chi, J., H¨ayh¨anen, E., and Jansen, B. J. (2024). Deus ex machina and personas from large language models: Investigating the composition of ai-generated persona descriptions. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 1–20.
53.Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., and Manning, C. D. (2024). Raptor: Recursive abstractive processing for tree-organized retrieval. arXiv preprint arXiv:2401.18059.
54.Scott, K. (2024). Behind the Tech. https://www.microsoft.com/en-us/behind-the-tech.
55.Shao, Z., Gong, Y., Shen, Y., Huang, M., Duan, N., and Chen, W. (2023). Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. arXiv preprint arXiv:2305.15294.
56.Shin, J., Hedderich, M. A., Rey, B. J., Lucero, A., and Oulasvirta, A. (2024). Understanding human-ai workflows for generating personas. In Proceedings of the 2024 ACM Designing Interactive Systems Conference, pages 757–781.
57.Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. (2024). Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36.
58.Su, D., Xu, Y., Yu, T., Siddique, F. B., Barezi, E. J., and Fung, P. (2020). Caire-covid: A question answering and query-focused multi-document summarization system for covid-19 scholarly information management. arXiv preprint arXiv:2005.03975.
59.Tan, Z., Zhao, X., and Wang, W. (2017). Representation learning of large-scale knowledge graphs via entity feature combinations. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM ’17, page 1777–1786, New York, NY, USA. Association for Computing Machinery.
60.Tang, Y. and Yang, Y. (2024). MultiHop-RAG: Benchmarking retrieval-augmented generation for multi-hop queries. arXiv preprint arXiv:2401.15391.
61.Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
62.Traag, V. A., Waltman, L., and Van Eck, N. J. (2019). From Louvain to Leiden: guaranteeing well-connected communities. Scientific Reports, 9(1).
63.Trajanoska, M., Stojanov, R., and Trajanov, D. (2023). Enhancing knowledge graph construction using large language models. ArXiv, abs/2305.04676.
64.Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2022). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509.
65.Wang, J., Liang, Y., Meng, F., Sun, Z., Shi, H., Li, Z., Xu, J., Qu, J., and Zhou, J. (2023a). Is chatgpt a good nlg evaluator? a preliminary study. arXiv preprint arXiv:2303.04048.
66.Wang, S., Khramtsova, E., Zhuang, S., and Zuccon, G. (2024). Feb4rag: Evaluating federated search in the context of retrieval augmented generation. arXiv preprint arXiv:2402.11891.
67.Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. (2022). Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
68.Wang, Y., Lipka, N., Rossi, R. A., Siu, A., Zhang, R., and Derr, T. (2023b). Knowledge graph prompting for multi-document question answering.
69.Xu, Y. and Lapata, M. (2021). Text summarization with latent queries. arXiv preprint arXiv:2106.00104.
70.Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. (2018). HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
71.Yao, J.-g., Wan, X., and Xiao, J. (2017). Recent advances in document summarization. Knowledge and Information Systems, 53:297–336.
72.Yao, L., Peng, J., Mao, C., and Luo, Y. (2023). Exploring large language models for knowledge graph completion.
73.Yates, A., Banko, M., Broadhead, M., Cafarella, M., Etzioni, O., and Soderland, S. (2007). TextRunner: Open information extraction on the web. In Carpenter, B., Stent, A., and Williams, J. D., editors, Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), pages 25–26, Rochester, New York, USA. Association for Computational Linguistics.
74.Yuan, X., Li, J., Wang, D., Chen, Y., Mao, X., Huang, L., Xue, H., Wang, W., Ren, K., and Wang, J. (2024). S-eval: Automatic and adaptive test generation for benchmarking safety evaluation of large language models. arXiv preprint arXiv:2405.14191.
75.Zhang, J. (2023). Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt. arXiv preprint arXiv:2304.11116.
76.Zhang, Y., Zhang, Y., Gan, Y., Yao, L., and Wang, C. (2024a). Causal graph discovery with retrieval-augmented generation based large language models. arXiv preprint arXiv:2402.15301.
77.Zhang, Z., Chen, J., and Yang, D. (2024b). Darg: Dynamic evaluation of large language models via adaptive reasoning graph. arXiv preprint arXiv:2406.17271.
78.Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. (2024). Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36.
79.Zhu, Y., Wang, X., Chen, J., Qiao, S., Ou, Y., Yao, Y., Deng, S., Chen, H., and Zhang, N. (2024). Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities.

Citation

MLA

Edge, D., et al. “From Local to Global: A Graph RAG Approach to Query-Focused Summarization”. arXiv, 2024, http://arxiv.org/abs/2404.16130v2.

APA

Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., & Larson, J. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv. http://arxiv.org/abs/2404.16130v2

Chicago

Edge, D., H. Trinh, N. Cheng, et al. 2024. “From Local to Global: A Graph RAG Approach to Query-Focused Summarization”. arXiv. http://arxiv.org/abs/2404.16130v2.

Harvard

Edge, D. et al. (2024) “From Local to Global: A Graph RAG Approach to Query-Focused Summarization”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2404.16130v2.

Vancouver

1. Edge D, Trinh H, Cheng N, Bradley J, Chao A, Mody A, Truitt S, Metropolitansky D, Ness RO, Larson J (2024) From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv

BibTeX

@article{edge2024from,
  title = {From Local to Global: A Graph RAG Approach to Query-Focused Summarization},
  author = {Edge, Darren and Trinh, Ha and Cheng, Newman and Bradley, Joshua and Chao, Alex and Mody, Apurva and Truitt, Steven and Metropolitansky, Dasha and Ness, Robert Osazuwa and Larson, Jonathan},
  year = {2024},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2404.16130v2},
  eprint = {2404.16130}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

From Local to Global: A Graph RAG Approach to Query-Focused Summarization

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — GraphRAG Knowledge Graph Indexing Pipeline

Knowl 2 — Hierarchical Community Summarization Algorithm

Knowl 3 — Map-Reduce Query-Focused Summarization for Global Sensemaking

Knowl 4 — Self-Reflection Gleaning for Entity Extraction

Knowl 5 — Adaptive Benchmarking for Sensemaking Question Generation

Knowl 6 — Claim-Based Evaluation Framework for Summarization Quality

Knowl 7 — Comparative Sensemaking Performance of GraphRAG vs Vector RAG