keyword
commonsense question answering
Commonsense question answering is a natural language processing task in which an artificial intelligence system answers questions that require everyday human knowledge and practical reasoning rather than explicitly stated facts. Unlike standard reading comprehension tasks that retrieve answers directly from a provided text, this task evaluates the ability of a system to infer implicit assumptions about the physical, social, and temporal dynamics of the world. Successfully addressing these problems typically requires computational systems to combine large language models with structured knowledge graphs or multi-step reasoning techniques to identify plausible outcomes and relationships that humans naturally understand.
2 items
JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering
Yueqing Sun, Qi Shi, Le Qi, Yu Zhang
Why you should read this
Proposes JointLK, a model that unites pretrained language models and graph neural networks through fine-grained bidirectional attention and dynamic node pruning to filter irrelevant knowledge and deliver interpretable commonsense question answering.
Existing KG-augmented models for common-sense question answering primarily focus on designing elaborate Graph Neural Networks (GNNs) to model knowledge graphs (KGs). However, they ignore (i) the effectively fusing and reasoning over question context representations and the KG representations, and (ii) automatically selecting relevant nodes from the noisy KGs during reasoning. In this paper, we propose a novel model, JointLK, which solves the above limitations through the joint reasoning of LM and GNN and the dynamic KGs pruning mechanism. Specifically, JointLK performs joint reasoning between LM and GNN through a novel dense bidirectional attention module, in which each question token attends on KG nodes and each KG node attends on question tokens, and the two modal representations fuse and update mutually by multi-step interactions. Then, the dynamic pruning module uses the attention weights generated by joint reasoning to prune irrelevant KG nodes recursively. We evaluate JointLK on the CommonsenseQA and OpenBookQA datasets, and demonstrate its improvements to the existing LM and LM+KG models, as well as its capability to perform interpretable reasoning¹.
Added
2026-09-26
STaR: Bootstrapping Reasoning With Reasoning
Eric Zelikman, Yuhuai Wu, Jesse Mu, Noah D. Goodman
Why you should read this
Demonstrates an iterative self-improvement cycle wherein a model generates rationales and continuously fine-tunes itself on the traces that yield correct answers, fundamentally linking alignment with reasoning capabilities.
Generating step-by-step "chain-of-thought" rationales improves language model performance on complex reasoning tasks like mathematics or commonsense question-answering. However, inducing language model rationale generation currently requires either constructing massive rationale datasets or sacrificing accuracy by using only few-shot inference. We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning. This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat. We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30 larger state-of-the-art language model on CommensenseQA. Thus, STaR lets a model improve itself by learning from its own generated reasoning.
Added
2026-05-08
License
Published with permission