Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

COCO-QA dataset

The COCO-QA dataset is a benchmark dataset used for training and evaluating visual question answering systems, which are artificial intelligence models designed to answer natural language questions based on the visual content of images. Introduced to advance multimodal vision-and-language research, the dataset was constructed by automatically transforming descriptive captions from the Microsoft Common Objects in Context repository into question-and-answer pairs using syntactic parsing and natural language processing algorithms. It contains over one hundred thousand questions paired with tens of thousands of real-world images, categorized into four specific domains: object recognition, color identification, numerical counting, and spatial location. A distinguishing feature of the dataset is that every question is paired with a single-word ground-truth answer, facilitating straightforward classification modeling and standardized performance evaluation across machine learning architectures.

1 item

Hierarchical Question-Image Co-Attention for Visual Question Answering

Hierarchical Question-Image Co-Attention for Visual Question Answering

Jiasen Lu, Jianwei Yang, Dhruv Batra, Devi Parikh

OrganizationsGeorgia Institute of TechnologyVirginia Tech

Why you should read this

Introduces a hierarchical co-attention model for visual question answering that jointly computes attention over relevant image regions and multi-level language structures across word, phrase, and question representations.

A number of recent works have proposed attention models for Visual Question Answering (VQA) that generate spatial maps highlighting image regions relevant to answering the question. In this paper, we argue that in addition to modeling "where to look" or visual attention, it is equally important to model "what words to listen to" or question attention. We present a novel co-attention model for VQA that jointly reasons about image and question attention. In addition, our model reasons about the question (and consequently the image via the co-attention mechanism) in a hierarchical fashion via a novel 1-dimensional convolution neural networks (CNN). Our model improves the state-of-the-art on the VQA dataset from 60.3% to 60.5%, and from 61.6% to 63.3% on the COCO-QA dataset. By using ResNet, the performance is further improved to 62.1% for VQA and 65.4% for COCO-QA.

Added

2026-09-24