Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

constrained text generation

Constrained text generation is a natural language processing task in which a language model produces text that strictly satisfies a predefined set of conditions or rules while maintaining fluency and coherence. These constraints can be lexical, such as requiring or forbidding specific words and phrases, structural, such as adhering to a target length or format schema, or logical, such as fulfilling combinations of positive and negative keyword requirements. Rather than relying solely on the unconstrained probability distribution of a language model, constrained text generation guides the generation or decoding process to systematically explore output paths that fulfill all specified requirements, making it essential for applications such as terminology-controlled machine translation, table-to-text generation, and keyword-guided writing.

1 item

NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics

NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics

Ximing Lu, Sean Welleck, Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lianhui Qin, Youngjae Yu, Rowan Zellers, Noah A. Smith, Yejin Choi

OrganizationsAllen Institute for AIUniversity of Washington

Why you should read this

Proposes an A*-inspired decoding algorithm with efficient lookahead heuristics that enables autoregressive language models to satisfy complex lexical constraints and achieve state-of-the-art performance across multiple text generation benchmarks without task-specific training data.

The dominant paradigm for neural text generation is left-to-right decoding from autoregressive language models. Constrained or controllable generation under complex lexical constraints, however, requires foresight to plan ahead for feasible future paths. Drawing inspiration from the A* search algorithm, we propose NEUROLOGIC A★esque,1 a decoding algorithm that incorporates heuristic estimates of future cost. We develop lookahead heuristics that are efficient for large-scale language models, making our method a drop-in replacement for common techniques such as beam search and top-k sampling. To enable constrained generation, we build on NEUROLOGIC decoding (Lu et al., 2021), combining its flexibility in incorporating logical constraints with A★esque estimates of future constraint satisfaction. Our approach outperforms competitive baselines on five generation tasks, and achieves new state-of-the-art performance on table-to-text generation, constrained machine translation, and keyword-constrained generation. The improvements are particularly notable on tasks that require complex constraint satisfaction or in few-shot or zero-shot settings. NEUROLOGIC A★esque illustrates the power of decoding for improving and enabling new capabilities of large-scale language models.

Added

2026-09-28