Built independently by an author, for readers. Read the story and support ChapterPal

keyword

deterministic quantization

Deterministic quantization is a process in representation learning and signal processing that maps continuous-valued data into discrete representations using a fixed, non-random rule, typically by assigning each continuous vector to its nearest or best-matching vector in a predefined codebook. Unlike stochastic quantization, which samples discrete tokens from a probability distribution, deterministic quantization guarantees that identical inputs always produce the exact same discrete outputs. While this approach provides straightforward, unperturbed, and reproducible mappings for generative modeling and data compression, its rigid nearest-neighbor selection mechanism can lead to training challenges such as codebook underutilization, where only a subset of the available discrete codebook entries is actively utilized.

1 item

Regularized Vector Quantization for Tokenized Image Synthesis

Regularized Vector Quantization for Tokenized Image Synthesis

Jiahui Zhang, Fangneng Zhan, Christian Theobalt, Shijian Lu

OrganizationsMax Planck Institute for InformaticsNanyang Technological University

Why you should read this

Proposes a dual-regularized vector quantization framework with a probabilistic contrastive loss that prevents codebook collapse and aligns training with stochastic sampling for superior image synthesis in autoregressive and diffusion models.

Quantizing images into discrete representations has been a fundamental problem in unified generative modeling. Predominant approaches learn the discrete representation either in a deterministic manner by selecting the best-matching token or in a stochastic manner by sampling from a predicted distribution. However, deterministic quantization suffers from severe codebook collapse and misalignment with inference stage while stochastic quantization suffers from low codebook utilization and perturbed reconstruction objective. This paper presents a regularized vector quantization framework that allows to mitigate above issues effectively by applying regularization from two perspectives. The first is a prior distribution regularization which measures the discrepancy between a prior token distribution and the predicted token distribution to avoid codebook collapse and low codebook utilization. The second is a stochastic mask regularization that introduces stochasticity during quantization to strike a good balance between inference stage misalignment and unperturbed reconstruction objective. In addition, we design a probabilistic contrastive loss which serves as a calibrated metric to further mitigate the perturbed reconstruction objective. Extensive experiments show that the proposed quantization framework outperforms prevailing vector quantization methods consistently across different generative models including auto-regressive models and diffusion models.

Added

2026-09-26