Built independently by an author, for readers. Read the story and support ChapterPal

keyword

transformer-based image compression

Transformer-based image compression is a machine learning approach for reducing the data size of digital images by utilizing transformer neural network architectures to encode, decode, and model visual information. Unlike traditional handcrafted codecs or convolutional neural network models that are constrained by local receptive fields, transformer-based methods leverage self-attention mechanisms to capture long-range spatial dependencies and global contextual relationships across an entire image. In an end-to-end learned compression framework, transformer modules are typically incorporated into the autoencoder transforms to extract compact latent representations or into the entropy modeling stage to accurately estimate probability distributions, thereby optimizing the trade-off between visual fidelity and the required bitrate.

1 item

Frequency-Aware Transformer for Learned Image Compression

Frequency-Aware Transformer for Learned Image Compression

Han Li, Shaohui Li, Wenrui Dai, Chenglin Li, Junni Zou, Hongkai Xiong

OrganizationsShanghai Jiao Tong UniversityTsinghua University

Why you should read this

Introduces a frequency-aware transformer architecture for learned image compression that captures directional image details through multiscale frequency decomposition, outperforming the VTM-12.1 standard codec by more than 13% in BD-rate across benchmark datasets.

Learned image compression (LIC) has gained traction as an effective solution for image storage and transmission in recent years. However, existing LIC methods are redundant in latent representation due to limitations in capturing anisotropic frequency components and preserving directional details. To overcome these challenges, we propose a novel frequency-aware transformer (FAT) block that for the first time achieves multiscale directional ananlysis for LIC. The FAT block comprises frequency-decomposition window attention (FDWA) modules to capture multiscale and directional frequency components of natural images. Additionally, we introduce frequency-modulation feed-forward network (FMFFN) to adaptively modulate different frequency components, improving rate-distortion performance. Furthermore, we present a transformer-based channel-wise autoregressive (T-CA) model that effectively exploits channel dependencies. Experiments show that our method achieves state-of-the-art rate-distortion performance compared to existing LIC methods, and evidently outperforms latest standardized codec VTM-12.1 by 14.5%, 15.1%, 13.0% in BD-rate on the Kodak, Tecnick, and CLIC datasets.

Added

2026-09-26