Back

VisualBERT: A Simple and Performant Baseline for Vision and Language

Liunian Harold LiMark YatskarDa YinCho-Jui HsiehKai-Wei Chang

article2019arXiv2,388 citations

Introduces VisualBERT, a simple unified Transformer framework that aligns image regions with text tokens through self-attention, establishing a performant multimodal baseline across benchmarks such as VQA, VCR, and NLVR2.

Listen

Modern artificial intelligence applications increasingly require systems to jointly interpret text and visual scenes, ranging from answering questions about images to validating statements against complex visual evidence. Historically, models designed for these multimodal tasks have relied on complex, highly customized architectures tailored to specific benchmarks, which limits flexibility and increases engineering overhead. The article addresses this challenge by evaluating whether a unified, streamlined framework based on self-attention can effectively align visual and textual concepts across diverse tasks.

The article demonstrates that VisualBERT—a unified model that feeds both image region features and text tokens into a shared stack of Transformer layers—serves as a versatile and high-performing baseline. The evaluation covers four distinct vision-and-language tasks: visual question answering, visual commonsense reasoning, natural language visual reasoning across paired images, and phrase-to-region grounding. The system is first pre-trained on roughly 100,000 captioned images using visually grounded language modeling objectives and then adapted to each target application using standard fine-tuning workflows.

The experimental findings show that the unified architecture matches or exceeds previous state-of-the-art benchmarks despite its simpler structure. Across visual question answering and visual reasoning benchmarks, the model consistently surpassed baseline systems under comparable visual feature settings, posting substantial double-digit gains over earlier specialized architectures on visual reasoning. Diagnostic analyses further reveal that the model implicitly learns to ground entities and resolve grammatical relationships between words and corresponding image regions without explicit supervision, progressively refining these alignments in its deeper layers. Ablation studies confirm that allowing early fusion—processing vision and text together throughout all layers rather than late in the pipeline—alongside pre-training on paired multimodal data are the most critical drivers of performance.

These results demonstrate that organizations can reduce architectural complexity and maintenance costs by consolidating multimodal processing into a single, standardized framework rather than maintaining bespoke systems for different vision-and-language tasks. Because the model demonstrates internal tracking of visual-linguistic alignments, it offers greater interpretability for downstream systems while lowering deployment risk and engineering timelines.

Organizations evaluating multimodal machine learning deployments should prioritize unified Transformer architectures that integrate visual and text representations early in the processing pipeline. Prior to operational deployment, teams should conduct task-specific pre-training and consider validating performance on larger caption datasets to maximize accuracy. Future work identified in the article includes expanding this unified approach to purely visual tasks such as scene graph generation.

The findings should be interpreted with awareness of the experimental boundaries: results depend on the quality of upstream object detectors used to extract visual proposals, and pre-training was limited to a standard caption repository. While confidence in the model's baseline performance across the tested domains is high, performance on novel visual domains or complex out-of-distribution inputs may vary depending on the visual feature extractors employed.

Allen Institute for AIPeking UniversityUniversity of California, Los Angelesbilingualencoder-decoderlanguage modelVisualBERTImageLanguage
arXiv: 1908.03557uclanlp/visualbert
Cover for VisualBERT: A Simple and Performant Baseline for Vision and Language

Abstract

We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks. VisualBERT consists of a stack of Transformer layers that implicitly align elements of an input text and regions in an associated input image with self-attention. We further propose two visually-grounded language model objectives for pre-training VisualBERT on image caption data. Experiments on four vision-and-language tasks including VQA, VCR, NLVR2, and Flickr30K show that VisualBERT outperforms or rivals with state-of-the-art models while being significantly simpler. Further analysis demonstrates that VisualBERT can ground elements of language to image regions without any explicit supervision and is even sensitive to syntactic relationships, tracking, for example, associations between verbs and image regions corresponding to their arguments.

Topics

bilingualencoder-decoderlanguage modelsyntactic structure

Keywords

VisualBERTImageLanguageModelPre-trainingLayerVisualVisionAttentionCOCO

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 A Joint Representation Model for Vision and Language
  • 3.1 Background
  • 3.2 VisualBERT
  • 3.3 Training VisualBERT
  • 4 Experiment
  • 4.1 VQA
  • 4.2 VCR
  • 4.3 NLVR2
  • 4.4 Flickr30K Entities
  • 5 Analysis
  • 5.1 Ablation Study
  • 5.2 Dissecting Attention Weights
  • 5.3 Qualitative Analysis
  • 6 Conclusion and Future Work
  • References
  • A VQA
  • B VCR
  • C NLVR2
  • D Flickr30K

Knowls

  1. Knowl 1 — VisualBERT Architecture and Input Representation

    model/method

    VisualBERT extends the BERT Transformer architecture to jointly model text and visual inputs through a unified multi-layer self-attention stack. The base model consists of a 12-layer bidirectional Transformer encoder with a hidden dimensionality of 768 and 12 attention heads, initialized with pre-trained BERTBASE\text{BERT}_{\text{BASE}} parameters.

    The input to VisualBERT is a sequence of text token embeddings EE and a sequence of visual region embeddings FF:

    1. Text Embeddings: For each subword token, the input embedding e∈Ee \in E is the sum of three embeddings: e=et+es+epe = e_t + e_s + e_p where ete_t is the token embedding, ese_s is the segment embedding identifying the text segment, and epe_p is the positional embedding indicating token position in the text.

    2. Visual Embeddings: For each bounding region proposal f∈Ff \in F generated by an object detector (e.g., Faster R-CNN or ResNeXt), the embedding is computed as: f=fo+fs+fpf = f_o + f_s + f_p where fof_o is the visual feature vector extracted by a convolutional neural network (projected via a linear layer into the 768-dimensional embedding space if necessary), fsf_s is a dedicated segment embedding indicating that the token belongs to the visual modality, and fpf_p is a positional embedding. When explicit token-region alignments are provided in the input, fpf_p is set to the sum of the position embeddings epe_p of the aligned words; otherwise, it is omitted or standard.

    Both EE and FF are concatenated and passed directly into the first Transformer layer (early fusion), enabling all self-attention layers to discover implicit alignments between language tokens and visual regions across the entire network depth.

  2. Knowl 2 — Three-Phase Training Framework of VisualBERT

    model/method

    VisualBERT is trained through a three-stage pipeline consisting of task-agnostic pre-training, task-specific pre-training, and downstream fine-tuning:

    1. Task-Agnostic Pre-Training: Conducted on paired image-caption data (specifically the COCO dataset with ~100k images and 5 captions each) using two visually grounded objectives:
    • Masked Language Modeling with Image Context: Randomly selected subwords in the text sequence are replaced with a special token ([MASK]), and the model is optimized via cross-entropy to predict the original subwords given both the unmasked text tokens and the full set of unmasked visual region features.
    • Sentence-Image Prediction: The model receives two captions and an image. The first caption describes the image; the second caption is either another genuine caption for the same image (50% probability) or a randomly sampled caption from the dataset (50% probability). The model predicts binary classification of whether the provided text segments match the image.

    1. Task-Specific Pre-Training: Prior to fine-tuning on a downstream dataset, VisualBERT undergoes an intermediate pre-training stage on the target task dataset using the masked language modeling with visual context objective (plus any task-relevant auxiliary objective) to adapt its representations to the domain of the target task.

    2. Task-Specific Fine-Tuning: A task-specific classification or grounding head is added to the Transformer outputs, and the network is trained end-to-end using supervised downstream task objectives.

  3. Knowl 3 — Performance of VisualBERT on Visual Question Answering (VQA 2.0)

    empirical result

    On the VQA 2.0 dataset (consisting of over 1 million open-ended questions grounded on COCO images), VisualBERT is framed as a classification task over the 3,129 most frequent answers. A [MASK] token is appended to the question text, and its final hidden representation is passed through a linear classifier optimized with cross-entropy against normalized multi-target ground-truth answer distributions. Visual features are derived from a ResNeXt-based Faster R-CNN pre-trained on Visual Genome.

    Model Test-Dev Accuracy (%) Test-Std Accuracy (%)
    Pythia v0.1 68.49 -
    Pythia v0.3 68.71 -
    VisualBERT w/o Early Fusion 68.18 -
    VisualBERT w/o COCO Pre-training 70.18 -
    VisualBERT 70.80 71.00
    Pythia v0.1 + VG + Other Data Aug. 70.01 70.24
    MCAN + VG 70.63 70.90
    MCAN + VG + Multiple Detectors 72.55 -
    MCAN + VG + Multiple Detectors + BERT 72.80 -
    MCAN + VG + Multiple Detectors + BERT + Ensemble 75.00 75.23

    Under direct, comparable experimental setups (same visual features and proposal counts without external question-answer data augmentation or model ensembles), full VisualBERT achieves 70.80% on Test-Dev and 71.00% on Test-Std, outperforming the Pythia baselines while utilizing a simpler unified Transformer architecture.

  4. Knowl 4 — Performance of VisualBERT on Visual Commonsense Reasoning (VCR)

    empirical result

    The Visual Commonsense Reasoning (VCR) benchmark requires answering questions (Q→AQ \to A) and predicting justifications/rationales (QA→RQA \to R), with an overall metric requiring both to be correct (Q→ARQ \to AR). Inputs use ResNet-50 visual features on provided gold bounding boxes and incorporate provided entity-region coreference alignments into the visual positional embeddings fpf_p.

    Model Q→AQ \to A QA→RQA \to R Q→ARQ \to AR
    Dev Test Dev Test Dev Test
    R2C 63.8 65.1 67.2 67.3 43.1 44.0
    B2T2 (Leaderboard) - 72.6 - 75.7 - 55.0
    VisualBERT w/o Early Fusion 70.1 - 71.9 - 50.6 -
    VisualBERT w/o COCO Pre-training 67.9 - 69.5 - 47.9 -
    VisualBERT 70.8 71.6 73.2 73.2 52.2 52.4

    VisualBERT without COCO pre-training outperforms the R2C baseline using identical data resources (67.9% vs 63.8% on Q→AQ \to A Dev; 47.9% vs 43.1% on Q→ARQ \to AR Dev). Full VisualBERT achieves 71.6% (Q→AQ \to A), 73.2% (QA→RQA \to R), and 52.4% (Q→ARQ \to AR) on the hidden test set, demonstrating that pre-training on COCO transfers effectively to movie scenes with distinct visual domains.

  5. Knowl 5 — Performance of VisualBERT on Natural Language for Visual Reasoning (NLVR2)

    empirical result

    The NLVR2 dataset tests whether a natural language statement is true or false with respect to a pair of images. VisualBERT models this by taking the caption and region proposals from both images into a single sequence, using separate segment embeddings to distinguish between the first image, the second image, and text. Features are extracted using Detectron with 144 region proposals per image.

    Model Dev (%) Test-P (%) Test-U (%) Test-U Cons (%)
    MaxEnt 54.1 54.8 53.5 12.0
    VisualBERT w/o Early Fusion 64.6 - - -
    VisualBERT w/o COCO Pre-training 63.5 - - -
    VisualBERT 67.4 67.0 67.3 26.9

    Full VisualBERT achieves 67.4% Dev accuracy, 67.0% Test-P accuracy, 67.3% Test-U accuracy, and 26.9% Test-U Consistency, substantially outperforming the MaxEnt baseline (54.1% Dev, 12.0% consistency). In preliminary experiments on the effect of region proposal count on NLVR2 Dev accuracy, using 9, 18, 36, 72, and 144 proposals per image yielded accuracies of 64.8%, 65.5%, 66.7%, 67.1%, and 67.4%, respectively.

  6. Knowl 6 — Performance of VisualBERT on Phrase Grounding (Flickr30K Entities)

    empirical result

    On Flickr30K Entities, models must ground phrase spans from image captions to corresponding bounding box proposals. In VisualBERT, an additional self-attention block is introduced for fine-tuning, and the average attention weights across all heads from the final subword token of each phrase to each candidate box proposal are used as alignment logits, trained with cross-entropy loss.

    Model Recall@1 Recall@5 Recall@10 Upper Bound
    Dev Test Dev Test Dev Test Dev Test
    BAN - 69.69 - 84.22 - 86.35 86.97 87.45
    VisualBERT w/o Early Fusion 70.33 - 84.53 - 86.39 - - -
    VisualBERT w/o COCO Pre-training 68.07 - 83.98 - 86.24 - 86.97 87.45
    VisualBERT 70.40 71.33 84.49 84.98 86.31 86.51 - -

    VisualBERT outperforms the Bilinear Attention Network (BAN) baseline, achieving a Test Recall@1 of 71.33% (vs. 69.69%). The ablation model without early fusion performs comparably to the full model (70.33% vs. 70.40% Recall@1 on Dev), indicating that a shallower interaction depth may be sufficient for phrase-to-region grounding compared to complex visual reasoning.

  7. Knowl 7 — Ablation Study on VisualBERT Design Components

    empirical result

    An ablation study on the NLVR2 development set (evaluating models trained with 36 region features per image) quantifies the importance of task-agnostic pre-training, early fusion, language initialization, and sentence-image prediction:

    Model Variant NLVR2 Dev Accuracy (%)
    VisualBERT (Full Model) 66.7
    C1: VisualBERT w/o Grounded Pre-training (COCO text-only) 63.9
    C1: VisualBERT w/o COCO Pre-training (no task-agnostic stage) 62.9
    C2: VisualBERT w/o Early Fusion (fusion only at final layer) 61.4
    C3: VisualBERT w/o BERT Initialization (random initialization) 64.7
    C4: VisualBERT w/o Objective 2 (no sentence-image matching) 64.9

    The primary performance drivers are:

    1. Early Fusion (C2): Combining text and visual features starting from layer 1 provides the largest gain (+5.3% over late fusion).
    2. Grounded Pre-Training (C1): Pre-training on paired multimodal data (COCO images and captions) is critical; skipping visual grounding during pre-training causes a 2.8% drop, and skipping COCO pre-training entirely causes a 3.8% drop.
    3. Initialization and Sentence Prediction (C3, C4): Pre-trained BERT initialization provides +2.0%, while the sentence-image binary matching objective provides +1.8%.

  8. Knowl 8 — Emergent Unsupervised Entity Grounding in Attention Heads

    empirical result

    Analysis of the 144 self-attention heads (12 layers ×\times 12 heads) in VisualBERT prior to task fine-tuning shows that the model learns implicit entity grounding during task-agnostic pre-training on COCO captions without explicit phrase-region alignment supervision.

    When evaluating against ground-truth phrase-to-box annotations on Flickr30K (by masking intra-text attention and assigning each entity mention to the bounding region receiving maximum attention weight):

    • In lower Transformer layers (layers 1–4), individual attention heads exhibit lower entity grounding accuracy and broader attention distributions.
    • Grounding accuracy progressively concentrates and peaks in higher Transformer layers (layers 10–12), where specific attention heads achieve over 45%–50% alignment accuracy.
    • Across higher layers, these top heads significantly exceed the heuristic baseline that always selects the bounding box with the highest object detector confidence (~17% accuracy).

  9. Knowl 9 — Cross-Modal Syntactic Dependency Grounding in VisualBERT

    empirical result

    Evaluating pre-trained VisualBERT on sentences parsed with a dependency parser (Stanford Dependency Formalism via AllenNLP) reveals that individual self-attention heads discover cross-modal syntactic alignments without supervision.

    Given two words connected by a directed dependency relation w1→rw2w_1 \xrightarrow{r} w_2 where one word has a ground-truth bounding box in Flickr30K, specific heads accurately route attention from the dependent word directly to the bounding region of the governing argument:

    • For core predicate-argument relationships—including prepositional object (pobj), nominal subject (nsubj), and direct object (dobj)—VisualBERT contains specialized heads where verbs systematically attend to the visual regions of their noun arguments.
    • For modifier relationships—including adjectival modifier (amod)—heads attend from adjectives to the corresponding visual object regions.
    • For every tested dependency relation, at least one attention head significantly outperforms the heuristic baseline of choosing the detection with the highest detector confidence.

Coverage note — None was omitted; all contributed architectural definitions, pre-training/fine-tuning methods, task benchmark results (VQA 2.0, VCR, NLVR2, Flickr30K), ablation experiments, and internal self-attention head analyses are fully captured.

References

  1. 1.Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In CVPR, 2018.
  2. 2.Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. VQA: Visual question answering. In ICCV, 2015.
  3. 3.Remi Cadene, Hedi Ben-Younes, Matthieu Cord, and Nicolas Thome. MUREL: Multimodal relational reasoning for visual question answering. In CVPR, 2019.
  4. 4.Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015.
  5. 5.Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. What does BERT look at? an analysis of BERT's attention. BlackboxNLP, 2019.
  6. 6.Marie-Catherine De Marneffe and Christopher D Manning. Stanford typed dependencies manual. Technical report, 2008.
  7. 7.Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, 2019.
  8. 8.Timothy Dozat and Christopher D Manning. Deep biaffine attention for neural dependency parsing. ICLR, 2017.
  9. 9.Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), 2018.
  10. 10.Ross Girshick, Ilija Radosavovic, Georgia Gkioxari, Piotr Dollár, and Kaiming He. Detectron. https://github.com/facebookresearch/detectron, 2018.
  11. 11.Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering. In CVPR, 2017.
  12. 12.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
  13. 13.Yu Jiang, Vivek Natarajan, Xinlei Chen, Marcus Rohrbach, Dhruv Batra, and Devi Parikh. Pythia v0. 1: the winning entry to the VQA challenge 2018. arXiv preprint arXiv:1807.09956, 2018.
  14. 14.Lu Jiasen, Batra Dhruv, Parikh Devi, and Lee Lee. ViLBERT: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. arXiv preprint arXiv:1908.02265, 2019.
  15. 15.Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei. Image retrieval using scene graphs. In CVPR, 2015.
  16. 16.Andrej Karpathy and Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In CVPR, 2015.
  17. 17.Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. ReferItGame: Referring to objects in photographs of natural scenes. In EMNLP, 2014.
  18. 18.Jin-Hwa Kim, Jaehyun Jun, and Byoung-Tak Zhang. Bilinear attention networks. In NeurIPS, 2018.
  19. 19.Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015.
  20. 20.Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual Genome: Connecting language and vision using crowdsourced dense image annotations. International Journal of Computer Vision, 123(1):32–73, 2017.
  21. 21.Linjie Li, Zhe Gan, Yu Cheng, and Jingjing Liu. Relation-aware graph attention network for visual question answering. ArXiv, abs/1903.12314, 2019.
  22. 22.Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014.
  23. 23.Will Norcliffe-Brown, Stathis Vafeias, and Sarah Parisot. Learning conditioned graph structures for interpretable visual question answering. In NeurIPS, 2018.
  24. 24.Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In NAACL-HLT, 2018.
  25. 25.Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In ICCV, 2015.
  26. 26.Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. OpenAI, 2018.
  27. 27.Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI, 2019.
  28. 28.Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NeurIPS, 2015.
  29. 29.Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 2015.
  30. 30.Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning. In NeurIPS, 2017.
  31. 31.Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual Captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACL, 2018.
  32. 32.Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards VQA models that can read. In CVPR, 2019.
  33. 33.Alane Suhr, Stephanie Zhou, Iris Zhang, Huajun Bai, and Yoav Artzi. A corpus for reasoning about natural language grounded in photographs. ACL, 2019.
  34. 34.Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. VideoBert: A joint model for video and language representation learning. arXiv preprint arXiv:1904.01766, 2019.
  35. 35.Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.
  36. 36.Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. ACL, 2019.
  37. 37.Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016.
  38. 38.Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. Stacked attention networks for image question answering. In CVPR, 2016.
  39. 39.Jun Yu, Jing Li, Zhou Yu, and Qingming Huang. Multimodal transformer with multi-view visual representation for image captioning. arXiv preprint arXiv:1905.07841, 2019a.
  40. 40.Zhou Yu, Jun Yu, Yuhao Cui, Dacheng Tao, and Qi Tian. Deep modular co-attention networks for visual question answering. In CVPR, 2019b.
  41. 41.Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. From recognition to cognition: Visual commonsense reasoning. In CVPR, 2019.

Citation

MLA
Li, L. H., et al. “VisualBERT: A Simple and Performant Baseline for Vision and Language”. arXiv, 2019, https://doi.org/10.48550/arxiv.1908.03557.
APA
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019). VisualBERT: A Simple and Performant Baseline for Vision and Language. arXiv. https://doi.org/10.48550/arxiv.1908.03557
Chicago
Li, L. H., M. Yatskar, D. Yin, C.-J. Hsieh, and K.-W. Chang. 2019. “VisualBERT: A Simple and Performant Baseline for Vision and Language”. Preprint, ArXiv. https://doi.org/10.48550/arxiv.1908.03557.
Harvard
Li, L.H. et al. (2019) “VisualBERT: A Simple and Performant Baseline for Vision and Language”. arXiv. Available at: https://doi.org/10.48550/arxiv.1908.03557.
Vancouver
1. Li LH, Yatskar M, Yin D, Hsieh C-J, Chang K-W (2019) VisualBERT: A Simple and Performant Baseline for Vision and Language. https://doi.org/10.48550/arxiv.1908.03557

BibTeX

@misc{https://doi.org/10.48550/arxiv.1908.03557,
  doi = {10.48550/ARXIV.1908.03557},
  url = {https://arxiv.org/abs/1908.03557},
  author = {Li, Liunian Harold and Yatskar, Mark and Yin, Da and Hsieh, Cho-Jui and Chang, Kai-Wei},
  keywords = {Computer Vision and Pattern Recognition (cs.CV), Computation and Language (cs.CL), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {VisualBERT: A Simple and Performant Baseline for Vision and Language},
  publisher = {arXiv},
  year = {2019},
  copyright = {arXiv.org perpetual, non-exclusive license}
}
Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors