Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

classifier-free guidance scale

The classifier-free guidance scale is a hyperparameter used during the inference phase of conditional diffusion models to control how strongly the generated output adheres to a conditioning input, such as a text prompt or class label. It operates by combining the predictions of a conditional model and an unconditional model evaluated within the same network, extrapolating the difference between them. A scale value of one corresponds to standard conditional generation, while higher values amplify the influence of the conditioning signal, thereby improving fidelity and prompt alignment at the expense of sample diversity. When set excessively high, the scale can cause over-saturation and visual artifacts, making it a key parameter for balancing quality, adherence, and variety in generative tasks.

1 item

FiT: Flexible Vision Transformer for Diffusion Model

FiT: Flexible Vision Transformer for Diffusion Model

Zeyu Lu, Zidong Wang, Di Huang, Chengyue Wu, Xihui Liu, Wanli Ouyang, Lei Bai

OrganizationsShanghai Artificial Intelligence LaboratoryShanghai Jiao Tong UniversityTsinghua UniversityUniversity of Hong KongUniversity of Sydney

Why you should read this

Proposes a flexible vision transformer architecture that treats images as variable-length token sequences using 2D rotary positional embeddings, enabling diffusion models to generate high-fidelity images across arbitrary resolutions and aspect ratios without cropping.

Nature is infinitely resolution-free. In the context of this reality, existing diffusion models, such as Diffusion Transformers, often face challenges when processing image resolutions outside of their trained domain. To overcome this limitation, we present the Flexible Vision Transformer (FiT), a transformer architecture specifically designed for generating images with unrestricted resolutions and aspect ratios. Unlike traditional methods that perceive images as static-resolution grids, FiT conceptualizes images as sequences of dynamically-sized tokens. This perspective enables a flexible training strategy that effortlessly adapts to diverse aspect ratios during both training and inference phases, thus promoting resolution generalization and eliminating biases induced by image cropping. Enhanced by a meticulously adjusted network structure and the integration of training-free extrapolation techniques, FiT exhibits remarkable flexibility in resolution extrapolation generation. Comprehensive experiments demonstrate the exceptional performance of FiT across a broad range of resolutions. Repository available at https://github.com/whlzy/FiT.

Added

2026-09-26