Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

semantic energy

Semantic energy is an uncertainty quantification and reliability metric in natural language processing and machine learning used to evaluate a language model confidence in generated text and identify potential errors or hallucinations. Unlike standard probability-based measures that evaluate surface-level token distributions after softmax normalization, semantic energy groups candidate outputs into clusters of equivalent meaning and evaluates their collective confidence directly from the model unnormalized logit representations using an energy-based formulation inspired by Boltzmann distributions. By aggregating energy values across semantic clusters rather than individual lexical variations, this approach measures the model inherent certainty regarding the underlying ideas, offering a robust signal for detecting ungrounded claims, synthetic text artifacts, and out-of-distribution generations.

1 item

Cross-Domain Detection of GPT-2-Generated Technical Text

Cross-Domain Detection of GPT-2-Generated Technical Text

Juan Diego Rodriguez, Todd Hay, David Gros, Zain Shamsi, Ravi Srinivasan

OrganizationsEn SoluciónUniversity of California, DavisUniversity of Texas at Austin

Why you should read this

Demonstrates that machine-generated technical text detectors can successfully transfer across distinct scientific domains with only a few hundred labeled examples, enabling effective identification of synthetic content and paragraph tampering in full-length research papers.

Machine-generated text presents a potential threat not only to the public sphere, but also to the scientific enterprise, whereby genuine research is undermined by convincing, synthetic text. In this paper we examine the problem of detecting GPT-2-generated technical research text. We first consider the realistic scenario where the defender does not have full information about the adversary's text generation pipeline, but is able to label small amounts of in-domain genuine and synthetic text in order to adapt to the target distribution. Even in the extreme scenario of adapting a physics-domain detector to a biomedical detector, we find that only a few hundred labels are sufficient for good performance. Finally, we show that paragraph-level detectors can be used to detect the tampering of full-length documents under a variety of threat models.

Added

2026-09-26