keyword
generative commonsense
Generative commonsense refers to the capability of an artificial intelligence model to compose coherent, plausible natural language text that reflects intuitive, everyday understanding of the physical and social world. Unlike discriminative commonsense benchmarks that evaluate reasoning through multiple-choice selection or classification, generative commonsense requires a system to actively construct valid scenarios, descriptions, or explanations based on given concepts while respecting real-world relational dynamics and constraints. This ability relies on synthesizing implicit knowledge that is frequently omitted from explicit text, including temporal order, physical causality, object affordances, human motivations, and understanding what states or events are logically implausible.
2 items
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge
Jiangjie Chen, Wei Shi, Ziquan Fu, Sijie Cheng, Lei Li, Yanghua Xiao
Why you should read this
Reveals a fundamental belief conflict in large language models where they correctly answer yes-or-no questions about negative commonsense facts yet fail to generate text incorporating that same negative knowledge due to pre-training reporting biases.
Large language models (LLMs) have been widely studied for their ability to store and utilize positive knowledge. However, negative knowledge, such as “lions don’t live in the ocean”, is also ubiquitous in the world but rarely mentioned explicitly in the text. What do LLMs know about negative knowledge? This work examines the ability of LLMs to negative commonsense knowledge. We design a constrained keywords-to-sentence generation task (CG) and a Boolean question-answering task (QA) to probe LLMs. Our experiments reveal that LLMs frequently fail to generate valid sentences grounded in negative commonsense knowledge, yet they can correctly answer polar yes-or-no questions. We term this phenomenon the belief conflict of LLMs. Our further analysis shows that statistical shortcuts and negation reporting bias from language modeling pre-training cause this conflict.
Added
2026-09-26
Fine-Grained Controllable Text Generation Using Non-Residual Prompting
Fredrik Carlsson, Joey Öhman, Fangyu Liu, Severine Verlinden, Joakim Nivre, Magnus Sahlgren
Why you should read this
Proposes a non-residual attention architecture that enables fine-grained steering of causal language models at arbitrary decoding steps without degrading model representations or requiring labeled training data.
The introduction of immensely large causal language models (CLMs) has rejuvenated the interest in open-ended text generation. However, controlling the generative process for these Transformer-based models is at large an unsolved problem. Earlier work has explored either plug-and-play decoding strategies or more powerful but blunt approaches such as prompting. There hence currently exists a trade-off between fine-grained control and the capability for more expressive high-level instructions. To alleviate this trade-off, we propose an encoder-decoder architecture that enables intermediate text prompts at arbitrary time steps. We propose a resource-efficient method for converting a pre-trained CLM into this architecture and demonstrate its potential in various experiments, including the novel task of contextualized word inclusion. Our method provides strong results in multiple experimental settings, proving itself to be both expressive and versatile.¹
Added
2026-09-26