Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

Closed-book QA

Closed-book question answering is a natural language processing task in which an artificial intelligence model answers questions relying solely on the knowledge stored in its internal parameters during training, without retrieving or referencing external documents at test time. Unlike open-book question answering, which supplies relevant text passages or search results alongside the query, the closed-book setting provides only the question itself. The model must recall, synthesize, and output the correct information entirely from its learned representations. As a result, this paradigm is frequently used to evaluate how effectively language models memorize and recall factual knowledge, as well as to measure their tendency to generate ungrounded or hallucinated answers when deprived of external reference context.

1 item

Language Models with Conformal Factuality Guarantees

Language Models with Conformal Factuality Guarantees

Christopher Mohri, Tatsunori Hashimoto

OrganizationsAllen Institute for AIUniversity of California, San Diego

Why you should read this

Proposes a conformal prediction framework that provides rigorous statistical correctness guarantees for black-box language model outputs by progressively pruning uncertain sub-claims while preserving most generated content.

Large language models demonstrate impressive capabilities across diverse tasks yet can generate content unfaithful to external knowledge sources.To enforce controllable generation,this work introduces post-generation techniques for constrained decoding crafted specifically as defensible guards.We introduce SENSE,a framework leveraging span-level evidence to align outputs with desired alignment criteria.Rather than enforcing token-level constraints directly over entire vocabularies,SENSE operates through differentiable integration within multimodal retrieval settings,and further demonstrates how regulatory signals interact.In extensive experiments spanning three datasets,two scientific benchmarks,guided narrative completion,and factor verification pipelines grounded via Wikidata concepts.

Added

2026-10-02