Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

decoding-time alignment

Decoding-time alignment refers to a set of techniques in natural language processing that steer a language model during the text generation phase to ensure its outputs comply with specified goals, safety guidelines, and human preferences. Unlike training-time approaches that alter the underlying parameters of a model through fine-tuning or reinforcement learning, decoding-time alignment operates during inference by modifying token selection, utilizing search algorithms, or scoring candidate continuations against external reward functions and programmatic constraints. This framework allows practitioners to dynamically enforce customizable rules, balance trade-offs among multiple objectives, and improve adherence to desired behaviors without the computational cost and rigidity of retraining the base model.

1 item

DeAL: Decoding-time Alignment for Large Language Models

DeAL: Decoding-time Alignment for Large Language Models

James Y. Huang, Sailik Sengupta, Daniele Bonadiman, Yi-An Lai, Arshit Gupta, Nikolaos Pappas, Saab Mansour, Katrin Kirchhoff, Dan Roth

OrganizationsAmazon Web ServicesUniversity of Southern California

Why you should read this

Proposes DeAL, a framework that aligns large language models at decoding time by framing generation as heuristic search, enabling fine-grained control over customizable and modular reward functions without requiring model fine-tuning.

Large Language Models (LLMs) are nowadays expected to generate content aligned with human preferences. Current work focuses on alignment at model training time, through techniques such as Reinforcement Learning with Human Feedback (RLHF). However, it is unclear if such methods are an effective choice to teach alignment objectives to the model. First, the inability to incorporate multiple, custom rewards and reliance on a model developer’s view of universal and static principles are key limitations. Second, the reliability of such approaches is also questionable (e.g. susceptibility to jailbreaking even after safety training). To address these issues, we propose DeAL, a framework that allows the user to customize reward functions and enables Decoding-time ALignment of LLMs. At its core, we view decoding as a heuristic-guided search process and facilitate the use of a wide variety of alignment objectives. Our experiments with programmatic constraints such as keyword and length constraints, and abstract alignment objectives such as harmlessness and helpfulness, show that we can DeAL with fine-grained trade-offs and improve adherence to alignment objectives. Lastly, we demonstrate that DeAL is largely complementary to existing alignment strategies, and can be effectively paired with RLHF and prompting techniques to achieve better alignment.

Added

2026-09-26