keyword
vision-language training
Vision-language training is a machine learning process in which computational models are trained to jointly interpret, align, and reason over visual data, such as images or videos, and textual information. This training typically utilizes large-scale datasets containing paired or interleaved visual and text sequences, optimizing shared or cross-modal neural network architectures through objectives like cross-modal contrastive learning, masked feature reconstruction, or autoregressive next-token prediction across modalities. By mapping visual tokens and linguistic representations into a common semantic space, vision-language training enables models to perform diverse multimodal tasks, including visual question answering, cross-modal retrieval, image captioning, and long-form video comprehension.
1 item
