Built independently by an author, for readers. Read the story and support ChapterPal

keyword

attribute-based controlled text generation

Attribute-based controlled text generation is a natural language processing technique that directs language models to produce text adhering to specific desired characteristics, such as sentiment, topic, emotion, style, or formality, while maintaining overall fluency and coherence. Unlike unconstrained text generation, which relies solely on an input prompt and standard probabilistic likelihood, this approach applies targeted steering mechanisms to satisfy explicit constraints. These mechanisms can operate during training, fine-tuning, prompting, or decoding, allowing systems to precisely regulate one or more thematic and stylistic properties in the generated output without degrading grammatical quality or core semantic meaning.

1 item

Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation

Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation

Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Mingfeng Xue, Boxing Chen, Jun Xie

OrganizationsAlibaba GroupNational University of Singapore

Why you should read this

Proposes a parameter-efficient framework that controls language models for single- and multi-attribute text generation by learning lightweight continuous prompts and prompt connectors while adding only 0.08% extra parameters to a frozen GPT-2.

Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes (e.g., emotions and topics). Existing work usually utilize fine-tuning or resort to extra attribute classifiers, yet suffer from increases in storage and inference time. To address these concerns, we explore attribute-based CTG in a parameter-efficient manner. In short, the proposed Tailor represents each attribute as a pre-trained continuous vector (i.e., single-attribute prompt), which guides the generation of a fixed pre-trained language model (PLM) to satisfy a pre-specified attribute. These prompts can be simply concatenated as a whole for multi-attribute CTG without any re-training. Nevertheless, this may raise problems of fluency downgrading and position sensitivity. To solve this, Tailor provides two solutions to enhance the combination. The former contains a multi-attribute prompt mask and a re-indexing position sequence to bridge the gap between the training (one single-attribute prompt for each task) and the testing stage (concatenating two prompts). The latter introduces a trainable prompt connector to further enhance the combinations. Experiments demonstrate that, only requiring 0.08% extra training parameters of the GPT-2, Tailor can achieve effective and general improvements on eleven attribute-specific generation tasks.

Added

2026-10-03