Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

Mask-Aware Transformer

A mask-aware transformer is a deep learning architecture designed for image inpainting that incorporates mask information directly into its attention mechanism to reconstruct missing or damaged visual regions. Unlike conventional vision transformers that model relationships across all input tokens uniformly, a mask-aware transformer uses dynamic masking to restrict or weight the aggregation of contextual information, ensuring that features are gathered predominantly from valid, uncorrupted pixels rather than empty or corrupted areas. By coupling this mask-guided self-attention with convolutional layers, the architecture efficiently captures both broad, long-range structures and fine local details, enabling high-resolution image synthesis and coherent restoration across large missing regions.

1 item

MAT: Mask-Aware Transformer for Large Hole Image Inpainting

MAT: Mask-Aware Transformer for Large Hole Image Inpainting

Wenbo Li, Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, Jiaya Jia

OrganizationsAdobeShanghai Artificial Intelligence LaboratoryThe Chinese University of Hong Kong

Why you should read this

Develops a mask-aware transformer that restricts attention computation to valid image tokens, enabling efficient high-resolution inpainting of large missing regions with high fidelity and diversity.

Recent studies have shown the importance of modeling long-range interactions in the inpainting problem. To achieve this goal, existing approaches exploit either standalone attention techniques or transformers, but usually under a low resolution in consideration of computational cost. In this paper, we present a novel transformer-based model for large hole inpainting, which unifies the merits of transformers and convolutions to efficiently process high-resolution images. We carefully design each component of our framework to guarantee the high fidelity and diversity of recovered images. Specifically, we customize an inpainting-oriented transformer block, where the attention module aggregates non-local information only from partial valid tokens, indicated by a dynamic mask. Extensive experiments demonstrate the state-of-the-art performance of the new model on multiple benchmark datasets. Code is released at this https URL.

Added

2026-09-26