Back

GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering

Drew A. HudsonChristopher D. Manning

article2019CVPR3,416 citations

Introduces GQA, a visual reasoning benchmark of 22 million compositional questions generated from real-world scene graphs with functional programs, enabling diagnostic evaluation of visual grounding, consistency, and multi-step reasoning while mitigating question bias.

Listen

The GQA dataset was created to overcome critical weaknesses in earlier visual question answering benchmarks, where models often relied on statistical shortcuts such as common object-color associations rather than genuine scene understanding, and where simple questions rarely demanded multi-step reasoning. The work therefore set out to produce a large-scale resource of real-world images paired with compositional questions that come with explicit semantic representations, allowing tighter control over answer distributions and finer-grained evaluation of model behavior.

The authors built the dataset from Visual Genome scene graphs covering 113,000 images. They normalized the graphs into a clean ontology, then used an engine of 524 linguistic patterns to generate 22 million questions whose semantics are captured by functional programs. These programs enabled rejection sampling that smoothed answer distributions both globally and locally while preserving real-world tendencies, yielding a final balanced set of 1.7 million questions. The same representations supported five new evaluation metrics that measure consistency across entailed questions, answer validity and plausibility, grounding of attention, and fidelity to the overall answer distribution.

Baseline and state-of-the-art models reached 42–54 percent accuracy, while human performance reached 89 percent. Even the strongest models contradicted themselves on roughly one in five entailed questions and produced implausible answers at least 5–15 percent of the time. Accuracy rose with textual question length yet fell with semantic depth, and models benefited substantially from object-level rather than spatial visual features. Transfer experiments showed that training on GQA improved results on the VQA dataset, indicating that the new questions retain realism while posing a stricter test.

These results demonstrate that current systems still fall short of coherent, grounded reasoning and that accuracy alone masks important failure modes. The dataset therefore supplies both a more reliable benchmark and direct supervision signals—functional programs and visual pointers—that can guide development of models with greater consistency and transparency.

Further progress will require architectures that integrate visual knowledge extraction with multi-step reasoning and that explicitly enforce consistency across related questions. Larger training regimes and richer scene-graph annotations are also likely to help. The main limitations are that questions are automatically generated rather than crowdsourced in free form and that performance remains bounded by the coverage and accuracy of the underlying scene graphs; both factors should be considered when interpreting absolute scores.

Stanford Universityenginefunctional programsgraph structuresGQA DatasetVisual ReasoningCompositional Question Answering
arXiv: 1902.09506
Cover for GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering

Abstract

We introduce GQA, a new dataset for real-world visual reasoning and compositional question answering, seeking to address key shortcomings of previous VQA datasets. We have developed a strong and robust question engine that leverages scene graph structures to create 22M diverse reasoning questions, all come with functional programs that represent their semantics. We use the programs to gain tight control over the answer distribution and present a new tunable smoothing technique to mitigate question biases. Accompanying the dataset is a suite of new metrics that evaluate essential qualities such as consistency, grounding and plausibility. An extensive analysis is performed for baselines as well as state-of-the-art models, providing fine-grained results for different question types and topologies. Whereas a blind LSTM obtains mere 42.1%, and strong VQA models achieve 54.1%, human performance tops at 89.3%, offering ample opportunity for new research to explore. We strongly hope GQA will provide an enabling resource for the next generation of models with enhanced robustness, improved consistency, and deeper semantic understanding for images and language.

Topics

enginefunctional programsgraph structureslong short term memory neural networksquestion answeringreasoning

Keywords

GQA DatasetVisual ReasoningCompositional Question AnsweringVisual Genome Scene GraphFunctional ProgramBias MitigationAnswer Distribution BalancingEvaluation MetricBaseline Performance GapLarge‑Scale Question Set

Table of Contents

  • 1. Introduction
  • 2. Related Work
  • 3.1. Scene Graph Normalization
  • 3.2. The Question Engine
  • 3.3. Functional Representation and Entailment
  • 3.4. Sampling and Balancing
  • 4. Analysis and Baseline Experiments
  • 4.1. Dataset Analysis and Comparison
  • 4.2. Baseline Experiments
  • 4.3. Transfer Performance
  • 4.4. New Evaluation Metrics
  • 5. Conclusion
  • 6. Acknowledgments
  • 7. Dataset Visualizations
  • 8. Dataset Balancing
  • 9. Baselines Implementation Details
  • 10. Further Diagnosis
  • 11. Comparison between GQA and VQA 2.0
  • 12. Scene Graph Normalization
  • Image Annotation
  • Image Question Answering

Knowls

  1. Knowl 1 — GQA Dataset Specification and Composition

    definition

    The GQA dataset is a large-scale visual reasoning benchmark built on real-world images from COCO and Flickr, formalized via Visual Genome scene graphs. The full raw dataset consists of 22,669,678 questions generated over 113,018 images (supplemented with 5,000 newly crowdsourced hidden scene graphs for testing), which is downsampled to a balanced subset of 1.7 million questions to eliminate conditional answer priors.

    The dataset features a vocabulary of 3,097 words and 1,878 candidate answers, split image-wise into 70% training, 10% validation, 10% test, and 10% challenge sets. Every question is characterized along two structural and semantic axes:

    • Structural Types (derived from the final program operation): query (open questions, 53%), verify (yes/no questions, 22%), choose (alternative selection questions, 12%), logical (boolean inference questions, 10%), and compare (comparison questions between objects, 3%).
    • Semantic Types (derived from the core question subject): relation (52%), attribute (28%), object (11%), category (6%), and global (scene-level attributes like weather or location, 3%).

    In terms of compositional complexity (the number of semantic reasoning operations), 6% of questions require 1 step, 42% require 2 steps, 44% require 3 steps, and 8% require 4 or more reasoning steps.

  2. Knowl 2 — Scene Graph Normalization and Ontology Creation

    model/method

    To enable programmatic question generation from Visual Genome annotations without linguistic noise, the scene graph normalization pipeline converts free-form annotations into a structured semantic ontology containing 2,690 classes (1,740 objects, 620 attributes, and 330 relations grouped into a hierarchy of 60 categories and subcategories).

    The normalization pipeline executes four sequential stages:

    1. Vocabulary Consolidation and Ontology Construction: Stop words and typos are removed, synonyms are merged, and rare or ambiguous concepts are filtered out. Vocabulary items are classified into semantic categories using word embedding distances followed by manual curation, adding grammatical metadata (part of speech, voice, plurality, synonyms).
    2. Edge Pruning: Unnatural or invalid relationship triplets (e.g., (woman, in, shirt) or (hand, hugging, bear)) are pruned using a combination of category-based rules, nn-gram frequency statistics from large text corpora, co-occurrence statistics, and manual filtering.
    3. Detector Verification: Because Visual Genome annotations are not exhaustive, object detectors trained on Visual Genome with low detection thresholds are used to conservatively verify the absence or uniqueness of objects before generating negative or existence questions.
    4. Positional and Property Augmentation: Scene graphs are enriched with absolute spatial positions (e.g., image margins), relative spatial relations (e.g., left/right, above/below), attribute-sharing edges (pairs sharing color, shape, or material), and scene-level global properties (weather and location inferred from object co-occurrences).

  3. Knowl 3 — Compositional Question Generation via Scene Graph Traversal

    model/method

    The GQA question engine synthesizes multi-step reasoning questions by traversing normalized scene graphs and instantiating 524 structural patterns (250 manual seed patterns and 274 patterns derived from templatizing VQA 1.0 questions) organized into 117 question groups. Each question is simultaneously generated as a natural language sentence and as an executable functional program composed of atomic operations (e.g., object selection, edge traversal, attribute filtering, boolean operations).

    Compositionality and natural language diversity are achieved through three mechanisms:

    • Direct vs. Indirect References: When an object is unique within the image, the engine uses a direct reference (e.g., the bear). When an object is ambiguous or to introduce multi-step reasoning, it constructs nested indirect references using attribute modifiers and relational paths (e.g., the animal behind the tree wearing a red coat).
    • Decoy Selection for Counterfactuals: For negative and logical questions (e.g., asking if a red apple is green, or if a person is eating cake when eating pizza), distractors are selected based on high co-occurrence likelihood with the subject within the scene context, while candidates that are semantically too close to the ground truth (e.g., pink vs. orange) are filtered out using an ontology exclusion list.
    • Probabilistic Pattern Variation: Structural patterns incorporate optional clauses [x][x] and alternate phrasing choices (x∣y)(x|y) that are sampled at random, along with lexical synonyms, ensuring diverse linguistic surface forms.

  4. Knowl 4 — Two-Level Tunable Distribution Smoothing Algorithm

    algorithm

    To mitigate question-conditional bias (where models guess answers from linguistic priors rather than visual evidence), the GQA balancing procedure applies rejection sampling across two semantic partition granularities: globally by question answer type (e.g., color), and locally by subject-answer type (e.g., apple-color). The procedure flattens the answer distribution head while preserving authentic relative frequency rankings.

    Input: Answer distribution counts C=[(a1,c1),(a2,c2),…,(ak,ck)]C = [(a_1, c_1), (a_2, c_2), \dots, (a_k, c_k)] sorted such that c1≥c2≥⋯≥ckc_1 \ge c_2 \ge \dots \ge c_k, maximum head-to-tail ratio bound bb, minimum adjacent ratio rmin⁡r_{\min}, maximum adjacent ratio rmax⁡r_{\max}
    Output: Smoothed answer counts C~=[(a~1,c~1),…,(a~k,c~k)]\tilde{C} = [(\tilde{a}_1, \tilde{c}_1), \dots, (\tilde{a}_k, \tilde{c}_k)]
    Initialize c~j=cj\tilde{c}_j = c_j for all j=1,…,kj = 1, \dots, k
    for i=1i = 1 to k−1k - 1 do
        Shead=∑j=1ic~jS_{\text{head}} = \sum_{j=1}^i \tilde{c}_j
        Stail=∑j=i+1kc~jS_{\text{tail}} = \sum_{j=i+1}^k \tilde{c}_j
        if Stail>0S_{\text{tail}} > 0 and (Shead/Stail)>b(S_{\text{head}} / S_{\text{tail}}) > b then
            scale=(b⋅Stail)/Shead\text{scale} = (b \cdot S_{\text{tail}}) / S_{\text{head}}
            for j=1j = 1 to ii do
                c~j=c~j⋅scale\tilde{c}_j = \tilde{c}_j \cdot \text{scale}
            end for
        end if
        ratio = c~i+1/c~i\tilde{c}_{i+1} / \tilde{c}_i
        if ratio < rmin⁡r_{\min} then
            c~i+1=c~i⋅rmin⁡\tilde{c}_{i+1} = \tilde{c}_i \cdot r_{\min}
        else if ratio > rmax⁡r_{\max} then
            c~i+1=c~i⋅rmax⁡\tilde{c}_{i+1} = \tilde{c}_i \cdot r_{\max}
        end if
    end for
    return C~\tilde{C}

    Applying this two-level smoothing increases the empirical entropy of the conditional answer distribution by 72% and reduces the dataset from 22M raw questions to a debiased balanced set of 1.7M questions.

  5. Knowl 5 — Visual Reasoning Consistency, Validity, and Plausibility Metrics

    definition

    GQA defines three complementary diagnostic metrics to evaluate reasoning properties beyond accuracy:

    • Consistency: Measures whether a model maintains logical coherence across interrelated questions. For any question qq answered correctly by the model from the set of all correctly answered questions QQ, let Eq={q1,q2,…,qn}E_q = \{q_1, q_2, \dots, q_n\} denote the set of entailed or equivalent questions derived from qq's functional program (e.g., answering What color is the apple? -> red entails that Is the apple red? -> yes and Is the apple green? -> no). Consistency is computed as the average model accuracy on the entailed sets across all q∈Qq \in Q: Consistency=1∣Q∣∑q∈Q1∣Eq∣∑q′∈EqI(a^q′=aq′∗)\text{Consistency} = \frac{1}{|Q|} \sum_{q \in Q} \frac{1}{|E_q|} \sum_{q' \in E_q} \mathbb{I}(\hat{a}_{q'} = a_{q'}^*) where a^q′\hat{a}_{q'} is the model prediction and aq′∗a_{q'}^* is the true answer for entailed question q′q'.

    • Validity: The percentage of questions where the predicted answer falls within the semantic domain of the question type (e.g., predicting a color word in response to a color query).

    • Plausibility: The percentage of predictions that are reasonable in the context of the question subject, defined as whether the subject-answer pair occurs at least once anywhere across the full dataset (e.g., predicting red or green for an apple is plausible, whereas purple or wooden is implausible).

  6. Knowl 6 — Answer Distribution Distance and Visual Attention Grounding Metrics

    definition

    GQA introduces distribution modeling and visual attention grounding metrics to diagnose shortcut learning:

    • Distribution Score: Quantifies the divergence between the true conditional answer distribution P(A∣Q)P(A \mid Q) and the model predicted distribution Q(A∣Q)Q(A \mid Q) using the Chi-Square (χ2\chi^2) statistic across answer categories kk: χ2=∑k(P(ak)−Q(ak))2P(ak)+ϵ\chi^2 = \sum_{k} \frac{(P(a_k) - Q(a_k))^2}{P(a_k) + \epsilon} where lower values indicate better alignment with the true distribution, demonstrating that the model learns infrequent tail answers rather than collapsing onto dominant majority classes.

    • Grounding Score: Evaluates whether an attention-based model focuses its visual reasoning on the image region relevant to the query. For each dataset instance, let rr represent the set of spatial locations or bounding box regions identified by ground-truth visual pointers for the question and answer entities. Grounding is the cumulative visual attention probability mass assigned by the model to region rr: Grounding=∑i∈rαi\text{Grounding} = \sum_{i \in r} \alpha_i where αi\alpha_i is the model's normalized attention weight at spatial location or bounding box ii.

  7. Knowl 7 — Benchmark Evaluation of VQA Models and Human Performance on GQA

    data/table

    Evaluation of baseline heuristics, unimodal models, multimodal architectures, and human upper bounds on the GQA test set demonstrates that visual question answering models struggle with compositional multi-step reasoning and consistency, falling far short of human performance.

    Metric Global Prior Local Prior CNN LSTM CNN+LSTM BottomUp MAC Humans
    Open 16.52 16.99 1.74 22.69 31.80 34.83 38.91 87.4
    Binary 42.99 47.53 36.05 61.90 63.26 66.64 71.23 91.2
    Query 16.52 16.99 1.55 22.69 31.80 34.83 38.91 87.4
    Compare 35.59 41.91 36.34 57.79 56.62 56.32 60.04 93.1
    Choose 17.45 26.58 0.85 57.15 61.40 66.56 70.59 94.3
    Logical 50.32 50.11 47.18 61.73 62.05 64.03 69.99 88.5
    Verify 53.40 58.80 47.02 65.78 67.00 71.45 75.45 90.1
    Global 24.70 20.19 8.64 27.22 56.57 60.29 60.82 92.3
    Object 49.96 54.00 47.33 74.33 75.90 78.45 81.49 88.1
    Attribute 34.89 42.67 22.66 48.28 50.91 53.88 59.82 90.7
    Relation 22.88 20.16 11.60 33.24 39.45 42.84 46.16 89.2
    Category 15.26 17.31 3.56 22.33 37.49 41.18 44.38 90.3
    Distribution (χ2\chi^2) 130.86 21.56 19.99 17.93 7.46 5.98 5.34 –
    Grounding (%) – – – – – 78.47 82.24 –
    Validity (%) 89.02 84.44 35.78 96.39 96.02 96.18 96.16 98.9
    Plausibility (%) 75.34 84.42 34.84 87.30 84.25 84.57 84.48 97.2
    Consistency (%) 51.78 54.34 62.40 68.68 74.57 78.71 81.59 98.4
    Accuracy (%) 28.93 31.31 17.82 41.07 46.55 49.74 54.06 89.3

    Blind language-only models (LSTM) achieve 41.07% accuracy overall but drop to 22.69% on open query questions, showing that GQA suppresses blind guessing. MAC achieves the highest model accuracy (54.06%) and consistency (81.59%), yet remains 35.24% behind human accuracy (89.30%) and 16.81% behind human consistency (98.40%).

  8. Knowl 8 — Domain Transfer Generalization Between GQA and VQA 2.0

    empirical result

    Cross-dataset transfer experiments between GQA and VQA 2.0 illustrate that GQA provides a rich, realistic proxy for natural visual question answering while imposing stronger reasoning demands:

    • GQA to VQA Transfer: A MAC model trained solely on GQA achieves 52.1% accuracy when evaluated on VQA 2.0 without fine-tuning, and improves to 60.5% after fine-tuning on VQA. For comparison, a CNN+LSTM baseline trained directly on VQA achieves 51.6%, and MAC trained directly on VQA achieves 68.3%.
    • VQA to GQA Transfer: A MAC model trained on VQA 2.0 achieves only 39.8% zero-shot accuracy on GQA, improving to 46.5% after fine-tuning on GQA.

    These results confirm that the vocabulary, compositional templates, and visual diversity of GQA transfer effectively to natural questions, whereas models trained on standard VQA benchmarks fail to generalize to the balanced, debiased multi-step reasoning structure of GQA.

  9. Knowl 9 — Linguistic and Structural Comparison Between GQA and VQA 2.0

    data/table

    A structural and linguistic head-to-head comparison between GQA and VQA 2.0 reveals that GQA contains substantially longer, more grammatically complex questions requiring multi-hop reasoning over relations and spatial configurations.

    Aspect VQA 2.0 GQA
    Question length (words) 6.2±1.96.2 \pm 1.9 7.9±3.17.9 \pm 3.1
    Verbs per question 1.4±0.61.4 \pm 0.6 1.6±0.71.6 \pm 0.7
    Nouns per question 1.9±0.91.9 \pm 0.9 2.5±1.02.5 \pm 1.0
    Adjectives per question 0.6±0.60.6 \pm 0.6 0.7±0.70.7 \pm 0.7
    Prepositions per question 0.5±0.60.5 \pm 0.6 1.0±1.01.0 \pm 1.0
    Relation questions (%) 19.5% 51.6%
    Spatial questions (%) 8.0% 22.4%
    Logical questions (%) 6.0% 19.0%
    Comparative questions (%) 1.0% 3.0%
    Compositional questions (%) 3.0% 52.0%

    GQA questions exhibit double the preposition density of VQA 2.0, more than 2.5 times the prevalence of relational questions (51.6% vs. 19.5%), nearly three times the spatial questions (22.4% vs. 8.0%), and a seventeen-fold increase in compositional multi-hop questions (52.0% vs. 3.0%).

  10. Knowl 10 — Impact of Semantic Reasoning Steps, Modality Representations, and Question Length on Reasoning Models

    empirical result

    Diagnostic ablations on the MAC architecture on GQA reveal the effects of question structure, reasoning depth, and input representations:

    • Textual vs. Semantic Question Length: Textual question word count correlates positively with model accuracy, likely because longer linguistic frames provide more contextual cues. In contrast, semantic question length (number of functional reasoning operations) correlates negatively with accuracy: single-operation questions attain >70%>70\% accuracy, whereas multi-hop questions requiring 2, 3, or ≥4\ge 4 operations drop to 50%50\%, 43%43\%, and <40%<40\% accuracy respectively.
    • Visual Input Representations: Using grid-level spatial features from a CNN achieves only 43% visual grounding and lower overall accuracy. Object-level features extracted via Faster R-CNN boost visual grounding to 82.24% and accuracy to 54.06%. Supplying oracle scene graph embeddings ('perfect sight') further increases model accuracy to ~80%.
    • Linguistic vs. Programmatic Questions: Training MAC on canonical functional programs yields 59.7% accuracy and 85.85% consistency, compared to 53.8% accuracy and 81.59% consistency on natural language questions. This gap highlights the additional reasoning challenge introduced by natural language phenomena such as long-range dependencies and lexical synonymy.

Coverage note — None was omitted; all primary dataset specifications, construction pipelines, balancing algorithms, evaluation metrics, comparative tables, transfer experiments, and model diagnostic ablations are fully covered.

References

  1. 1.Google books ngram corpus http://books.google.com/ngrams/. 17
  2. 2.A. Agrawal, D. Batra, and D. Parikh. Analyzing the behavior of visual question answering models. In EMNLP, pages 1955–1960, 2016. 1, 2
  3. 3.A. Agrawal, D. Batra, D. Parikh, and A. Kembhavi. Don’t just assume; look and answer: Overcoming priors for visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4971–4980, 2018. 1, 2
  4. 4.A. Agrawal, J. Lu, S. Antol, M. Mitchell, C. L. Zitnick, D. Parikh, and D. Batra. VQA: Visual question answering. International Journal of Computer Vision, 123(1):4–31, 2017. 1, 2, 4
  5. 5.P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould, and L. Zhang. Bottom-up and top-down attention for image captioning and VQA. arXiv preprint arXiv:1707.07998, 2017. 1, 7, 8, 14, 15
  6. 6.J. Andreas, M. Rohrbach, T. Darrell, and D. Klein. Neural module networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 39–48, 2016. 1
  7. 7.Y. Attali and M. Bar-Hillel. Guess where: The position of correct answers in multiple-choice test items as a psychometric variable. Journal of Educational Measurement, 40(2):109–128, 2003. 2
  8. 8.A. Das, H. Agrawal, L. Zitnick, D. Parikh, and D. Batra. Human attention in visual question answering: Do humans and deep networks look at the same regions? Computer Vision and Image Understanding, 163:90–100, 2017. 1, 2
  9. 9.J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255. Ieee, 2009. 14
  10. 10.A. Fukui, D. H. Park, D. Yang, A. Rohrbach, T. Darrell, and M. Rohrbach. Multimodal compact bilinear pooling for visual question answering and visual grounding. Conference on Empirical Methods in Natural Language Processing (EMNLP), 2016. 1
  11. 11.Y. Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6325–6334, 2017. 1, 2, 3, 6, 16
  12. 12.D. A. Hudson and C. D. Manning. Compositional attention networks for machine reasoning. International Conference for Representation Learning (ICLR), 2018. 1, 6, 7, 8, 14, 15
  13. 13.A. Jabri, A. Joulin, and L. van der Maaten. Revisiting visual question answering baselines. In European conference on computer vision, pages 727–739. Springer, 2016. 1
  14. 14.U. Jain, Z. Zhang, and A. G. Schwing. Creativity: Generating diverse questions using variational autoencoders. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5415–5424, 2017. 3
  15. 15.J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, and R. Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1988–1997, 2017. 1, 2, 3, 7
  16. 16.J. Johnson, R. Krishna, M. Stark, L.-J. Li, D. Shamma, M. Bernstein, and L. Fei-Fei. Image retrieval using scene graphs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3668–3678, 2015. 3
  17. 17.K. Kafle and C. Kanan. An analysis of visual question answering algorithms. In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 1983–1991. IEEE, 2017. 2, 3
  18. 18.K. Kafle and C. Kanan. Visual question answering: Datasets, algorithms, and future challenges. Computer Vision and Image Understanding, 163:3–20, 2017. 1, 2, 16
  19. 19.D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 15
  20. 20.R. Krishna, Y. Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y. Kalantidis, L.-J. Li, D. A. Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International Journal of Computer Vision, 123(1):32–73, 2017. 1, 2, 3, 4, 17
  21. 21.H. O. Lancaster and E. Seneta. Chi-square distribution. Encyclopedia of biostatistics, 2, 2005. 8
  22. 22.Q. Li, Q. Tao, S. Joty, J. Cai, and J. Luo. VQA-E: Explaining, elaborating, and enhancing your answers for visual questions. arXiv preprint arXiv:1803.07464, 2018. 2
  23. 23.T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft COCO: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 4, 17
  24. 24.A. Mahendru, V. Prabhu, A. Mohapatra, D. Batra, and S. Lee. The promise of premise: Harnessing question premises in visual question answering. arXiv preprint arXiv:1705.00601, 2017. 3
  25. 25.M. Malinowski and M. Fritz. A multi-world approach to question answering about real-world scenes based on uncertain input. In Advances in neural information processing systems, pages 1682–1690, 2014. 3
  26. 26.J. Millman, C. H. Bishop, and R. Ebel. An analysis of test-wiseness. Educational and Psychological Measurement, 25(3):707–726, 1965. 2
  27. 27.J. J. Mondak and B. C. Davis. Asked and answered: Knowledge levels when we won’t take ‘don’t know’ for an answer. Political Behavior, 23(3):199–224, 2001. 2
  28. 28.N. Mostafazadeh, I. Misra, J. Devlin, M. Mitchell, X. He, and L. Vanderwende. Generating natural questions about an image. arXiv preprint arXiv:1603.06059, 2016. 3
  29. 29.D. H. Park, L. A. Hendricks, Z. Akata, A. Rohrbach, B. Schiele, T. Darrell, and M. Rohrbach. Multimodal explanations: Justifying decisions and pointing to the evidence. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 2
  30. 30.J. Pennington, R. Socher, and C. Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543, 2014. 14
  31. 31.S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems, pages 91–99, 2015. 14, 16, 17
  32. 32.Y. Rubner, C. Tomasi, and L. J. Guibas. The earth mover’s distance as a metric for image retrieval. International journal of computer vision, 40(2):99–121, 2000. 6
  33. 33.A. Suhr, S. Zhou, I. Zhang, H. Bai, and Y. Artzi. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:1811.00491, 2018. 2
  34. 34.D. Teney, P. Anderson, X. He, and A. van den Hengel. Tips and tricks for visual question answering: Learnings from the 2017 challenge. arXiv preprint arXiv:1708.02711, 2017. 2, 15
  35. 35.D. Teney, L. Liu, and A. van den Hengel. Graphstructured representations for visual question answering. arXiv preprint, 2017. 2
  36. 36.B. Thomee, D. A. Shamma, G. Friedland, B. Elizalde, K. Ni, D. Poland, D. Borth, and L.-J. Li. Yfcc100m: The new data in multimedia research. arXiv preprint arXiv:1503.01817, 2015. 4, 17
  37. 37.A. Torralba and A. A. Efros. Unbiased look at dataset bias. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1521–1528. IEEE, 2011. 2
  38. 38.Z. Yang, X. He, J. Gao, L. Deng, and A. Smola. Stacked attention networks for image question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 21–29, 2016. 1
  39. 39.P. Zhang, Y. Goyal, D. Summers-Stay, D. Batra, and D. Parikh. Yin and yang: Balancing and answering binary visual questions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5014–5022, 2016. 1, 2
  40. 40.S. Zhang, L. Qu, S. You, Z. Yang, and J. Zhang. Automatic generation of grounded visual questions. arXiv preprint arXiv:1612.06530, 2016. 3
  41. 41.Y. Zhu, O. Groth, M. Bernstein, and L. Fei-Fei. Visual7W: Grounded question answering in images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4995–5004, 2016. 1, 2, 3

Citation

MLA
Hudson, D. A., and C. D. Manning. “GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering”. arXiv, 2019, http://arxiv.org/abs/1902.09506v3.
APA
Hudson, D. A., & Manning, C. D. (2019). GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. arXiv. http://arxiv.org/abs/1902.09506v3
Chicago
Hudson, D. A., and C. D. Manning. 2019. “GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering”. arXiv. http://arxiv.org/abs/1902.09506v3.
Harvard
Hudson, D.A. and Manning, C.D. (2019) “GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1902.09506v3.
Vancouver
1. Hudson DA, Manning CD (2019) GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. arXiv

BibTeX

@article{hudson2019gqa,
  title = {GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering},
  author = {Hudson, Drew A. and Manning, Christopher D.},
  year = {2019},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1902.09506v3},
  eprint = {1902.09506}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE