Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

factual hallucinations

Factual hallucinations refer to outputs generated by artificial intelligence models, such as large language models, that present fabricated, incorrect, or unverifiable information as true facts. In natural language processing, these errors occur when a model produces text that directly contradicts established real-world knowledge or fails to align with provided reference materials, such as retrieved source documents. Such inaccuracies can manifest as invented events, false relationships, or fabricated citations, often delivered with high grammatical fluency and apparent confidence. Identifying and mitigating factual hallucinations is essential for ensuring the truthfulness, consistency, and practical reliability of automated generative systems.

1 item

RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models

RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models

Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, Tong Zhang

OrganizationsNewsBreakUniversity of Illinois Urbana-Champaign

Why you should read this

Presents RAGTruth, a large-scale word-level hallucination benchmark of nearly 18,000 manually annotated retrieval-augmented generations that enables smaller language models to match GPT-4 in detecting and mitigating factual errors.

Retrieval-augmented generation (RAG) has become a main technique for alleviating hallucinations in large language models (LLMs). Despite the integration of RAG, LLMs may still present unsupported or contradictory claims to the retrieved contents. In order to develop effective hallucination prevention strategies under RAG, it is important to create benchmark datasets that can measure the extent of hallucination. This paper presents RAGTruth, a corpus tailored for analyzing word-level hallucinations in various domains and tasks within the standard RAG frameworks for LLM applications. RAGTruth comprises nearly 18,000 naturally generated responses from diverse LLMs using RAG. These responses have undergone meticulous manual annotations at both the individual case and word levels, incorporating evaluations of hallucination intensity. We not only benchmark hallucination frequencies across different LLMs, but also critically assess the effectiveness of several existing hallucination detection methodologies. We show that using a high-quality dataset such as RAGTruth, it is possible to finetune a relatively small LLM and achieve a competitive hallucination detection performance when compared to the existing prompt-based approaches using state-of-the-art LLMs such as GPT-4. Furthermore, the finetuned model can effectively mitigate hallucination in LLM responses.

Added

2026-09-26