keyword
reasoning graphs
A reasoning graph is a structured representation that models the logical pathways, intermediate conclusions, and relational dependencies involved in solving a complex problem or verifying a claim. In such a graph, nodes represent entities, concepts, facts, or discrete thought states, while edges capture the semantic, causal, or inferential connections that link them together. By structuring deductions as an interconnected network rather than a purely linear sequence, reasoning graphs facilitate compositional multi-step inference, enable structured evidence aggregation across diverse sources, and enhance the transparency and explainability of artificial intelligence systems by making their underlying reasoning processes explicit and inspectable.
2 items
MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models
Yilin Wen, Zifeng Wang, Jimeng Sun
Why you should read this
Proposes a plug-and-play prompting framework that combines external knowledge graphs with large language models to construct transparent reasoning pathways and reduce hallucinations in complex question answering.
Large language models (LLMs) have achieved remarkable performance in natural language understanding and generation tasks. However, they often suffer from limitations such as difficulty in incorporating new knowledge, generating hallucinations, and explaining their reasoning process. To address these challenges, we propose a novel prompting pipeline, named MindMap, that leverages knowledge graphs (KGs) to enhance LLMs' inference and transparency. Our method enables LLMs to comprehend KG inputs and infer with a combination of implicit and external knowledge. Moreover, our method elicits the mind map of LLMs, which reveals their reasoning pathways based on the ontology of knowledge. We evaluate our method on diverse question & answering tasks, especially in medical domains, and show significant improvements over baselines. We also introduce a new hallucination evaluation benchmark and analyze the effects of different components of our method. Our results demonstrate the effectiveness and robustness of our method in merging knowledge from LLMs and KGs for combined inference. To reproduce our results and extend the framework further, we make our codebase available at https://github.com/wyl-willing/MindMap.
Added
2026-10-01
SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables
Xinyuan Lu, Liangming Pan, Qian Liu, Preslav Nakov, Min-Yen Kan
Why you should read this
Introduces SCITAB, a benchmark of 1.2K expert-verified scientific claims derived from real research papers that tests whether language models can perform compositional and numerical reasoning directly over scientific tables.
Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an over-reliance on text-based evidence. We present SCITAB, a challenging evaluation dataset consisting of 1.2K expert-verified scientific claims that 1) originate from authentic scientific publications and 2) require compositional reasoning for verification. The claims are paired with evidence-containing scientific tables annotated with labels. Through extensive evaluations, we demonstrate that SCITAB poses a significant challenge to state-of-the-art models, including table-based pretraining models and large language models. All models except GPT-4 achieved performance barely above random guessing. Popular prompting techniques, such as Chain-of-Thought, do not achieve much performance gains on SCITAB. Our analysis uncovers several unique challenges posed by SCITAB, including table grounding, claim ambiguity, and compositional reasoning. Our codes and data are publicly available at https://github.com/XinyuanLu00/SciTab.
Added
2026-09-26