Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

late fusion

Late fusion is an integration strategy in multimodal machine learning where separate models independently process individual data modalities to generate distinct predictions or high-level decisions before combining them into a final output. Unlike early fusion, which merges raw features or low-level representations at the input stage, late fusion operates at the decision level using aggregation mechanisms such as majority voting, score averaging, learned weighting, or secondary meta-classifiers. This modular approach allows specialized architectures to be independently trained and optimized for heterogeneous data types, such as text, audio, video, and tabular data. Furthermore, late fusion offers practical robustness and flexibility, enabling systems to more easily accommodate missing, asynchronous, or varying-quality modalities during training and inference.

1 item

Provable Dynamic Fusion for Low-Quality Multimodal Data

Provable Dynamic Fusion for Low-Quality Multimodal Data

Qingyang Zhang, Haitao Wu, Changqing Zhang, Qinghua Hu, Huazhu Fu, Joey Tianyi Zhou, Xi Peng

Why you should read this

Establishes theoretical generalization bounds for dynamic decision-level multimodal integration and introduces Quality-aware Multimodal Fusion to prevent performance degradation on noisy and low-quality inputs.

The inherent challenge of multimodal fusion is to precisely capture the cross-modal correlation and flexibly conduct cross-modal interaction. To fully release the value of each modality and mitigate the influence of low-quality multimodal data, dynamic multimodal fusion emerges as a promising learning paradigm. Despite its widespread use, theoretical justifications in this field are still notably lacking. Can we design a provably robust multimodal fusion method? This paper provides theoretical understandings to answer this question under a most popular multimodal fusion framework from the generalization perspective. We proceed to reveal that several uncertainty estimation solutions are naturally available to achieve robust multimodal fusion. Then a novel multimodal fusion framework termed Quality-aware Multimodal Fusion (QMF) is proposed, which can improve the performance in terms of classification accuracy and model robustness. Extensive experimental results on multiple benchmarks can support our findings.

Added

2026-09-26