Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

regularized vector quantization

Regularized vector quantization is a discrete representation learning method used in generative modeling to map continuous latent features to discrete codebook vectors while imposing explicit regularization constraints. Building upon standard vector quantization frameworks, it incorporates regularizing objectives to mitigate common optimization issues such as codebook collapse, low codebook utilization, and discrepancies between training and inference stages. These regularization strategies typically involve matching predicted token distributions with prior distributions to ensure balanced codebook usage, introducing controlled stochasticity during quantization, and employing calibrated contrastive or reconstruction objectives to maintain high representation fidelity. By balancing deterministic code selection with stochastic exploration, regularized vector quantization improves training stability, codebook efficiency, and sample reconstruction quality in generative architectures such as autoregressive and diffusion models.

1 item

Regularized Vector Quantization for Tokenized Image Synthesis

Regularized Vector Quantization for Tokenized Image Synthesis

Jiahui Zhang, Fangneng Zhan, Christian Theobalt, Shijian Lu

OrganizationsMax Planck Institute for InformaticsNanyang Technological University

Why you should read this

Proposes a dual-regularized vector quantization framework with a probabilistic contrastive loss that prevents codebook collapse and aligns training with stochastic sampling for superior image synthesis in autoregressive and diffusion models.

Quantizing images into discrete representations has been a fundamental problem in unified generative modeling. Predominant approaches learn the discrete representation either in a deterministic manner by selecting the best-matching token or in a stochastic manner by sampling from a predicted distribution. However, deterministic quantization suffers from severe codebook collapse and misalignment with inference stage while stochastic quantization suffers from low codebook utilization and perturbed reconstruction objective. This paper presents a regularized vector quantization framework that allows to mitigate above issues effectively by applying regularization from two perspectives. The first is a prior distribution regularization which measures the discrepancy between a prior token distribution and the predicted token distribution to avoid codebook collapse and low codebook utilization. The second is a stochastic mask regularization that introduces stochasticity during quantization to strike a good balance between inference stage misalignment and unperturbed reconstruction objective. In addition, we design a probabilistic contrastive loss which serves as a calibrated metric to further mitigate the perturbed reconstruction objective. Extensive experiments show that the proposed quantization framework outperforms prevailing vector quantization methods consistently across different generative models including auto-regressive models and diffusion models.

Added

2026-09-26