Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

Input clarification ensembling

Input clarification ensembling is an uncertainty quantification method for language models that generates multiple clarified variations of an ambiguous or under-specified input and aggregates the model predictions produced across those variations. By resolving the potential interpretations of an initial prompt into distinct, concrete variants, this technique allows practitioners to decompose total predictive uncertainty into aleatoric uncertainty, which stems from inherent ambiguity in the input data, and epistemic uncertainty, which reflects the model's own knowledge limitations. The approach can be applied to standard pre-trained models without modifying model parameters or architecture, providing a structured mechanism to determine whether predictive variance is caused by missing contextual details in the query or gaps in the model's learned knowledge.

1 item

Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling

Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling

Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, Yang Zhang

OrganizationsMassachusetts Institute of TechnologyMIT-IBM Watson AI LabUniversity of California, Santa Barbara

Why you should read this

Proposes input clarification ensembling, a practical framework that separates large language model uncertainty into input ambiguity and model knowledge deficits without modifying model parameters or training procedures.

Uncertainty decomposition refers to the task of decomposing the total uncertainty of a predictive model into aleatoric (data) uncertainty, resulting from inherent randomness in the data-generating process, and epistemic (model) uncertainty, resulting from missing information in the model’s training data. In large language models (LLMs) specifically, identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability, but remains an important open research question. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling, which can be applied to any pre-trained LLM. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions. We show that, when aleatoric uncertainty arises from ambiguity or under-specification in LLM inputs, this approach makes it possible to factor an (un-clarified) LLM’s predictions into separate aleatoric and epistemic terms, using a decomposition similar to the one employed by Bayesian neural networks. Empirical evaluations demonstrate that input clarification ensembling provides accurate and reliable uncertainty quantification on several language processing tasks. Code and data are available at https://github.com/UCSB-NLP-Chang/llm_uncertainty.

Added

2026-10-01