Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

adversarial inputs

Adversarial inputs are specially crafted or perturbed pieces of data designed to intentionally deceive a machine learning model into producing incorrect, unexpected, or harmful outputs. In artificial intelligence systems, these inputs can take various forms, such as subtle modifications added to images to trigger misclassifications, or carefully structured prompts engineered to bypass safety guardrails and manipulate large language models into generating prohibited or toxic content. By exploiting vulnerabilities, statistical blind spots, or alignment flaws within a model learned representations, adversarial inputs allow researchers and practitioners to expose safety risks, evaluate robustness, and develop defenses to protect systems against malicious exploitation.

1 item

Detoxifying Large Language Models via Knowledge Editing

Detoxifying Large Language Models via Knowledge Editing

Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, Huajun Chen

OrganizationsAnt GroupMicrosoftMinistry of EducationNational University of SingaporeNUS-NCS Joint LabSoutheast UniversityWestlake UniversityZhejiang University

Why you should read this

Presents the SafeEdit benchmark and a single-instance knowledge editing method, DINM, to directly modify toxic model parameters rather than merely suppressing their activations, effectively detoxifying large language models without degrading their general performance.

This paper investigates using knowledge editing techniques to detoxify Large Language Models (LLMs). We construct a benchmark, SafeEdit, which covers nine unsafe categories with various powerful attack prompts and equips comprehensive metrics for systematic evaluation. We conduct experiments with several knowledge editing approaches, indicating that knowledge editing has the potential to detoxify LLMs with a limited impact on general performance efficiently. Then, we propose a simple yet effective baseline, dubbed Detoxifying with Intraoperative Neural Monitoring (DINM), to diminish the toxicity of LLMs within a few tuning steps via only one instance. We further provide an in-depth analysis of the internal mechanism for various detoxifying approaches, demonstrating that previous methods like SFT and DPO may merely suppress the activations of toxic parameters, while DINM mitigates the toxicity of the toxic parameters to a certain extent, making permanent adjustments. We hope that these insights could shed light on future work of developing detoxifying approaches and the underlying knowledge mechanisms of LLMs¹.

Added

2026-10-02