Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

toxification direction

The toxification direction is a vector in the internal representation space of a language model that captures the shift from standard or benign text generation toward the generation of offensive, toxic, or harmful content. It is typically identified by measuring the divergence in contextual representations and attention layer activations between unconstrained generation and generation guided by toxicity-inducing prompts. By isolating this directional trajectory within the model information stream, internal activations can be steered along the opposite vector, enabling the model to reduce harmful outputs and achieve detoxification without requiring extensive fine-tuning or supplementary decoding components.

1 item

Self-Detoxifying Language Models via Toxification Reversal

Self-Detoxifying Language Models via Toxification Reversal

Chak Tou Leong, Yi Cheng, Jiashuo Wang, Jian Wang, Wenjie Li

OrganizationsDepartment of ComputingHong Kong Polytechnic University

Why you should read this

Proposes an inference-time self-detoxification method that steers pretrained language models away from harmful text by identifying and reversing toxification representations within attention layers without requiring model fine-tuning or external classifiers.

Language model detoxification aims to minimize the risk of generating offensive or harmful content in pretrained language models (PLMs) for safer deployment. Existing methods can be roughly categorized as finetuning-based and decoding-based. However, the former is often resource-intensive, while the latter relies on additional components and potentially compromises the generation fluency. In this paper, we propose a more lightweight approach that enables the PLM itself to achieve “self-detoxification”. Our method is built upon the observation that prepending a negative steering prompt can effectively induce PLMs to generate toxic content. At the same time, we are inspired by the recent research in the interpretability field, which formulates the evolving contextualized representations within the PLM as an information stream facilitated by the attention layers. Drawing on this idea, we devise a method to identify the toxification direction from the normal generation process to the one prompted with the negative prefix, and then steer the generation to the reversed direction by manipulating the information movement within the attention layers. Experimental results show that our approach, without any fine-tuning or extra components, can achieve comparable performance with state-of-the-art methods.

Added

2026-10-03