keyword

VQA datasets

Visual question answering datasets are structured collections of visual media, such as images or videos, paired with natural language questions and corresponding ground-truth answers designed to train and benchmark multimodal artificial intelligence systems. These datasets enable models to bridge computer vision and natural language processing by requiring algorithms to interpret visual scenes, comprehend linguistic inquiries, and generate or select accurate textual responses. They encompass a wide variety of question formats and reasoning tasks, ranging from basic object recognition and attribute identification to complex spatial, causal, and common-sense reasoning. Depending on their design, these datasets can be created using synthetic imagery, curated real-world photographs, or user-generated mobile captures, coupled with crowdsourced, spoken, or machine-generated queries. Researchers rely on them to measure algorithmic accuracy, evaluate model generalization, and drive advancements in automated visual assistants, image retrieval, and human-computer interaction.

1 item

VizWiz Grand Challenge: Answering Visual Questions from Blind People

Danna Gurari, Qing Li, Abigale J. Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, Jeffrey P. Bigham

OrganizationsCarnegie Mellon UniversityUniversity of Colorado BoulderUniversity of RochesterUniversity of Science and Technology of ChinaUniversity of Texas at Austin

Why you should read this

Introduces VizWiz, a dataset of over 31,000 real-world visual questions from blind users that challenges visual question answering models to handle conversational queries, imperfect mobile photos, and unanswerable prompts in genuine assistive settings.

The study of algorithms to automatically answer visual questions currently is motivated by visual question answering (VQA) datasets constructed in artificial VQA settings. We propose VizWiz, the first goal-oriented VQA dataset arising from a natural VQA setting. VizWiz consists of over 31,000 visual questions originating from blind people who each took a picture using a mobile phone and recorded a spoken question about it, together with 10 crowdsourced answers per visual question. VizWiz differs from the many existing VQA datasets because (1) images are captured by blind photographers and so are often poor quality, (2) questions are spoken and so are more conversational, and (3) often visual questions cannot be answered. Evaluation of modern algorithms for answering visual questions and deciding if a visual question is answerable reveals that VizWiz is a challenging dataset. We introduce this dataset to encourage a larger community to develop more generalized algorithms that can assist blind people.

Source

https://arxiv.org/pdf/1802.08218

Added

2026-09-25