Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

probability distribution representations

Probability distribution representations are machine learning feature embeddings that map input data into probability distributions within a latent space instead of fixed, deterministic point vectors. By modeling representations as distributions typically characterized by statistical parameters such as mean and variance, this approach explicitly captures uncertainty, ambiguity, and multi-target semantic relationships inherent in complex or multimodal data. Unlike traditional vector embeddings that rely on standard point-wise distance metrics, distribution-based representations allow models to measure similarity and alignment using probabilistic divergences and statistical distances, thereby preserving richer semantic variability and improving robustness in downstream learning and reasoning tasks.

1 item

MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model

MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model

Yatai Ji, Junjie Wang, Yuan Gong, Lin Zhang, Yanru Zhu, Hongfa Wang, Jiaxing Zhang, Tetsuya Sakai, Yujiu Yang

OrganizationsInternational Digital Economy AcademyTencentTsinghua UniversityWaseda University

Why you should read this

Proposes a vision-language pre-training framework that models multimodal features as Gaussian distributions instead of deterministic points to capture inter- and intra-modal semantic uncertainty across downstream tasks like visual reasoning and image-text retrieval.

Multimodal semantic understanding often has to deal with uncertainty, which means the obtained messages tend to refer to multiple targets. Such uncertainty is problematic for our interpretation, including inter- and intra-modal uncertainty. Little effort has studied the modeling of this uncertainty, particularly in pre-training on unlabeled datasets and fine-tuning in task-specific downstream datasets. In this paper, we project the representations of all modalities as probabilistic distributions via a Probability Distribution Encoder (PDE) by utilizing sequence-level interactions. Compared to the existing deterministic methods, such uncertainty modeling can convey richer multimodal semantic information and more complex relationships. Furthermore, we integrate uncertainty modeling with popular pre-training frameworks and propose suitable pre-training tasks: Distribution-based Vision-Language Contrastive learning (D-VLC), Distribution-based Masked Language Modeling (D-MLM), and Distribution-based Image-Text Matching (D-ITM). The fine-tuned models are applied to challenging downstream tasks, including image-text retrieval, visual question answering, visual reasoning, and visual entailment, and achieve state-of-the-art results.

Added

2026-09-26