LLaVA-OneVision: Easy Visual Task Transfer

Bo LiYuanhan ZhangDong GuoRenrui ZhangFeng LiHao ZhangKaichen ZhangPei-Yuan ZhangYanwei LiZiwei Liu

article2024Trans. Mach. Learn. Res.2,993 citations

Presents LLaVA-OneVision, an open multimodal model that advances performance across single-image, multi-image, and video scenarios simultaneously through visual task transfer from images to videos.

Listen

Building general-purpose artificial intelligence assistants requires models that can seamlessly understand multiple visual formats, including individual photographs, sequences of related images, and continuous video streams. Historically, open-source large multimodal models have specialized in a single visual domain, often suffering significant performance degradation when applied across different formats. Concurrently, top-tier proprietary models dominate cross-scenario capabilities, leaving an open-source gap for an efficient, unified architecture capable of handling complex visual tasks across various operational contexts.

The article introduces and evaluates LLaVA-OneVision, an open-source family of large multimodal models designed to push the state-of-the-art across single-image, multi-image, and video understanding within a single, unified framework. Specifically, the authors aimed to demonstrate that careful data curation and a unified visual representation scheme enable strong transfer learning across distinct modalities without degrading core image reasoning capabilities.

To achieve this, the authors implemented a streamlined architecture connecting a pre-trained visual encoder (SigLIP) to a language model (Qwen-2) via a two-layer projection module. They introduced an adaptive representation strategy that caps the maximum visual token budget across single images (up to 7,290 tokens), multiple images (up to 8,748 tokens across up to 12 images), and video sequences (up to 6,272 tokens across up to 32 frames). The training pipeline followed a staged curriculum learning approach: initial language-image alignment, followed by high-quality synthetic knowledge injection (utilizing 4 million samples, of which 99.8% were synthetic), single-image instruction tuning on 3.2 million samples, and finally multi-modal fine-tuning across 1.6 million balanced mixed-modality samples. Models were evaluated across three parameter scales (0.5B, 7B, and 72B) across dozens of public academic and conversational benchmarks using standardized zero-shot evaluation protocols.

The findings confirm that LLaVA-OneVision establishes a new baseline for open multimodal systems, outperforming prior open models across all three vision scenarios. First, the 72B model achieves performance situated between commercial benchmarks GPT-4V and GPT-4o, notably surpassing GPT-4V on specialized single-image tasks such as document analysis (91.3% on DocVQA) and mathematical reasoning (67.5% on MathVista). Second, cross-modality tuning yielded massive improvements on multi-image tasks, where the 72B model reached 91.6% on MathVerse and 94.9% on SciVerse out-of-domain benchmarks. Third, video understanding transferred effectively from static image training, enabling competitive performance on benchmarks like VideoMME (66.2%) without requiring dedicated video architectures. Finally, the framework exhibited strong emergent cross-scenario capabilities not explicitly seen during training, such as interpreting multi-camera self-driving video, tracking visual markers in dynamic videos, and translating static images into detailed video generation instructions.

These results carry significant strategic implications for enterprise AI deployment. Organizations can reduce computational and operational complexity by deploying a single, modular open-source model capable of addressing diverse document, video, and multi-view inspection workflows rather than managing fragmented, task-specific pipelines. The finding that 99.8% synthetic pre-training data drove major knowledge gains suggests a cost-effective path around expensive, legally risky public web data scraping. Furthermore, the availability of lightweight variants (0.5B and 7B) enables deployment on resource-constrained edge devices while preserving high OCR and perceptual reasoning capabilities.

Decision-makers and engineering teams should consider adopting the LLaVA-OneVision framework as an open baseline for applications requiring multi-image reasoning, graphical user interface automation, and video QA. When designing internal pipelines, teams should adopt staged curriculum training and balance visual token allocation across modalities. For critical production workflows requiring highly open-ended, multi-turn conversational chat in the wild, organizations should conduct targeted pilot validations and implement additional preference alignment, as a noticeable performance gap relative to frontier proprietary models like GPT-4o remains in complex visual dialogue.

Confidence in these findings is supported by standardized zero-shot evaluation across standardized benchmarks using the LMMs-Eval framework. However, several constraints exist: training was executed under fixed compute budgets without exhaustive hyperparameter tuning across all sub-components, proprietary synthetic datasets created via commercial APIs cannot be fully redistributed, and models rely heavily on language backbone scaling to solve reasoning-intensive visual tasks.

arXiv: 2408.03326LLaVA-VL/LLaVA-NeXT
  • Paper: Improved Baselines with Visual Instruction Tuning, Haotian Liu et al. (2024). Reading LLaVA-1.5 provides the foundational design principles of visual instruction tuning and data efficiency that LLaVA-OneVision directly builds upon and extends.
Cover for LLaVA-OneVision: Easy Visual Task Transfer

Abstract

We present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series. Our experimental results demonstrate that LLaVA-OneVision is the first single model that can simultaneously push the performance boundaries of open LMMs in three important computer vision scenarios: single-image, multi-image, and video scenarios. Importantly, the design of LLaVA-OneVision allows strong transfer learning across different modalities/scenarios, yielding new emerging capabilities. In particular, strong video understanding and cross-scenario capabilities are demonstrated through task transfer from images to videos.

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 Modeling
  • 3.1 Network Architecture
  • 3.2 Visual Representations
  • 4 Data
  • 4.1 High-Quality Knowledge
  • 4.2 Visual Instruction Tuning Data
  • 5 Training Strategies
  • 6 Experimental Results
  • 6.1 Single-Image Benchmarks
  • 6.2 Multi-Image Benchmarks
  • 6.3 Video Benchmarks
  • 7 Emerging Capabilities with Task Transfer
  • 8 Conclusions
  • References
  • A Development Roadmap from LLaVA-NeXT to LLaVA-OneVision
  • B Author Contributions
  • C Implmenetation Details
  • C.1 Token Strategy for Mixed-Modality Data
  • C.2 Language Templates and Special Tokens
  • D Evaluation Steers Development
  • D.1 Post-Evaluation as a Development Tool
  • D.2 Improving Model Performance on Key Scenarios
  • D.3 Evaluation Task Information
  • E Data Curation Roadmap of LLaVA-NeXT Series
  • E.1 Single-Image Data Curation
  • E.2 OneVision Data Curation
  • E.3 Detailed Dataset Statistics
  • E.4 Policy Information and Reproducibility

Knowls

  1. Knowl 1 — LLaVA-OneVision Network Architecture and Autoregressive Formulation

    model/method

    LLaVA-OneVision is an open large multimodal model (LMM) family designed to process single-image, multi-image, and video inputs within a unified architecture.

    The model consists of three core components:

    1. Language Model (fϕf_\phi): Qwen-2 parameterized by ϕ\phi, available in 0.5B, 7B, and 72B parameter configurations.
    2. Vision Encoder (gψg_\psi): SigLIP (SO400M) parameterized by ψ\psi, which maps an input visual instance XvX_v (an image crop, a full image, or a video frame) into visual feature representations Zv=gψ(Xv)Z_v = g_\psi(X_v). Features from the grid before and after the final Transformer layer are utilized.
    3. Projection Module (pθp_\theta): A two-layer multilayer perceptron (MLP) parameterized by θ\theta that projects visual features ZvZ_v into the LLM word embedding space as visual tokens Hv=pθ(Zv)H_v = p_\theta(Z_v).

    For an output sequence Xa=(x1,x2,,xL)X_a = (x_1, x_2, \dots, x_L) of length LL conditioned on visual input XvX_v and instruction text XqX_q, the conditional probability is modeled autoregressively as:

    p(XaXv,Xq)=i=1Lp(xiXv,Xq,<i,Xa,<i)p(X_a \mid X_v, X_q) = \prod_{i=1}^L p(x_i \mid X_v, X_{q,<i}, X_{a,<i})

    where Xq,<iX_{q,<i} and Xa,<iX_{a,<i} denote the instruction and answer tokens generated prior to step ii.

    In text tokenization, visual inputs are marked with a special <image> token (represented internally by token index 200-200). In multi-image scenarios, multiple <image> tokens are interleaved within the prompt text at the respective image locations. In video scenarios, a single <image> token is prepended at the sequence start to designate the full video sequence.

  2. Knowl 2 — Higher AnyRes Visual Representation and Token Allocation Strategy Across Modalities

    model/method

    LLaVA-OneVision employs a flexible visual representation strategy called Higher AnyRes with Bilinear Interpolation that standardizes token representations across single-image, multi-image, and video modalities so that the maximum sequence length per sample remains comparable (~6,000 to ~8,700 tokens).

    Single-Image AnyRes

    An image is dynamically decomposed into a grid of a×ba \times b spatial crops of native resolution 384×384384 \times 384 pixels (each processed by SigLIP into T=729T = 729 visual tokens) plus one low-resolution global base image. The unconstrained visual token length is L=(a×b+1)×TL = (a \times b + 1) \times T. To prevent sequence explosion while retaining fine-grained resolution, token pooling via 2D bilinear interpolation reduces the token count per crop to TnewT_{\text{new}} if LL exceeds a maximum threshold τ\tau:

    Tnew={τa×b+1if L>τTif LτT_{\text{new}} = \begin{cases} \left\lfloor\frac{\tau}{a \times b + 1}\right\rfloor & \text{if } L > \tau \\ T & \text{if } L \le \tau \end{cases}

    In LLaVA-OneVision, spatial configurations (a,b)(a, b) are chosen from grid partitions up to 6×66 \times 6, allowing up to 9 crops plus 1 base crop (AnyResMax-9), resulting in a maximum of (1+9)×729=7,290(1 + 9) \times 729 = 7,290 tokens per image.

    Multi-Image Representation

    Each individual image in an interleaved sequence is resized to fit into a 384×384384 \times 384 frame using zero-padding to preserve aspect ratio. After passing through the SigLIP encoder, zero-padding tokens are stripped away. Up to 12 images per sequence are supported without multi-crop partitioning, yielding at most 12×729=8,74812 \times 729 = 8,748 visual tokens.

    Video Representation

    Video frames are sampled (up to 32 frames per video at 1 FPS) and resized to 384×384384 \times 384 base resolution. Each frame feature map is downsampled using a 2×22 \times 2 bilinear interpolation from 729 tokens to 196 tokens per frame. With 32 frames, this caps video visual tokens at 32×196=6,27232 \times 196 = 6,272 tokens.

  3. Knowl 3 — Multi-Stage Curriculum Training Pipeline for LLaVA-OneVision

    model/method

    LLaVA-OneVision is trained via a stage-wise curriculum learning pipeline with progressive sequence lengths, resolutions, and parameter unfreezing across four operational phases:

    1. Stage-1: Language-Image Alignment:

      • Data: LCS 558K dataset.
      • Visual Input: Base resolution 384×384384 \times 384 (729 tokens per sample).
      • Trainable Parameters: 2-layer MLP projector only (pθp_\theta).
      • Optimization: 1 epoch; global batch size 512; projector learning rate 1×1031 \times 10^{-3}.
    2. Stage-1.5: High-Quality Knowledge Learning:

      • Data: 4.0M high-quality synthetic and OCR/document samples.
      • Visual Input: AnyRes with spatial configurations up to 5 crops (max 729×5=3,645729 \times 5 = 3,645 tokens).
      • Trainable Parameters: Full model (Vision Encoder gψg_\psi, Projector pθp_\theta, LLM fϕf_\phi).
      • Optimization: 1 epoch; global batch size 512 (for 0.5B LLM) or 256 (for 7B/72B LLM); learning rates are LRLLM=LRproj=1×105\text{LR}_{\text{LLM}} = \text{LR}_{\text{proj}} = 1 \times 10^{-5} and LRvision=2×106\text{LR}_{\text{vision}} = 2 \times 10^{-6} (5×5\times smaller for the vision encoder).
    3. Stage-2 Phase 1: Single-Image Visual Instruction Tuning:

      • Data: 3.2M curated single-image visual instruction samples.
      • Visual Input: AnyRes with configurations up to 6×66 \times 6 (max 729×10=7,290729 \times 10 = 7,290 tokens).
      • Trainable Parameters: Full model.
      • Optimization: 1 epoch; batch size 512 (0.5B) or 256 (7B/72B); LRLLM=1×105\text{LR}_{\text{LLM}} = 1 \times 10^{-5}, LRvision=2×106\text{LR}_{\text{vision}} = 2 \times 10^{-6}. Yields intermediate checkpoint LLaVA-OV (SI).
    4. Stage-2 Phase 2: OneVision Mixed-Modality Tuning:

      • Data: 1.6M mixed-modality samples (800K single-image, 560K multi-image, 350K video).
      • Visual Input: Modality-balanced AnyRes representations (max ~7,290 tokens).
      • Trainable Parameters: Full model.
      • Optimization: 1 epoch; batch size 512 (0.5B) or 256 (7B/72B); LRLLM=1×105\text{LR}_{\text{LLM}} = 1 \times 10^{-5}, LRvision=2×106\text{LR}_{\text{vision}} = 2 \times 10^{-6}. Yields final checkpoint LLaVA-OV.
  4. Knowl 4 — Stage-1.5 High-Quality Knowledge Dataset Composition

    data/table

    To enhance model capabilities without low-quality web scrapes, Stage-1.5 injects knowledge using a 4.0M sample dataset wherein 99.8% of the data is synthetic. The composition comprises three primary categories:

    1. Re-Captioned Detailed Description Data (3.5M samples): Synthesized using LLaVA-NeXT-34B acting as an early-stage self-improvement generator on image sources from COCO118K, BLIP558K, and CC3M.
    2. Document and OCR Data (1.1M samples): Composed of the Text Reading subset (100K) from UReader rendered via PDFs, combined with SynDOG English and Chinese datasets (1.0M).
    3. Chinese and Language Data (235K samples): 92K detailed Chinese image captions generated via Azure OpenAI GPT-4V on ShareGPT4V images, paired with 143K pure language instruction samples from Evo-Instruct to maintain natural language reasoning balance.
  5. Knowl 5 — Visual Instruction Tuning Datasets: Single-Image 3.2M and OneVision 1.6M

    data/table

    LLaVA-OneVision visual instruction tuning is structured into two stages with distinct dataset compositions:

    1. Single-Image Instruction Mixture (3.2M samples)

    Organized into five balanced functional categories:

    • General QA (36.1%, ~1.14M samples): Includes Vision FLAN (186.1K), LLaVA-158K (158K), Image Textualization (99.6K), ShareGPT4V (92K), Cambrian-filtered (83.1K), VQAv2 (82.8K), ALLaVA Instruct (70K), AOKVQA (66.2K), ShareGPT4o (57.3K), LLaVA-Wild (54.5K), RefCOCO (50.6K), COCO Caption (20K), ST-VQA (17.2K), Visual7W (14.4K), WebSight (10K), VisText (10K), TallyQA (9.9K), Hateful Memes (8.5K), VizWiz (6.6K), ScienceQA (5.0K), IconQA (2.5K), VSR (2.2K), InterGPS (1.3K), CLEVR (700), VQARAD (313).
    • Doc / Chart / Screen (20.6%, ~647K samples): UReader QA (252.9K), UReader Caption (91.4K), RoBUT WikiSQL (75K), RoBUT WTQ (38.2K), UReader KG (37.6K), Chart2Text (27K), DVQA (20K), ChartQA (18.3K), UReader IE (17.3K), Screen2Words (15.7K), AI2D InternVL (12.4K), DocVQA (10.2K), RoBUT SQA (8.5K), AI2D GPT-4V detailed (4.9K), Infographic VQA (4.4K), AI2D Original (3.2K), VisualMRC (3.0K), HiTab (2.5K), LRV Chart (1.8K), TQA (1.4K), FigureQA (1.0K), Diagram Image2Text (300).
    • Math / Reasoning (20.1%, ~632K samples): MAVIS Data Engine (100K), MAVIS MCollect (87.4K), Visual Genome (86.4K), GQA (72.1K), Geo170K QA (67.8K), Geo170K Align (60.3K), TabMWP (45.2K), MathQA (29.8K), GeoQA+ (17.2K), UniGeo (12.0K), LRV Normal (10.5K), Geometry3K MathV360K (9.7K), GeoMVerse (9.3K), Super-CLEVR (8.7K), CLEVR-Math (5.3K), MapQA (5.2K), Geometry3K (2.1K), RAVEN (2.1K), GEOS (508).
    • Pure Language Data (14.3%, ~450K samples): Magpie Pro L3 MT (150K), Magpie Pro L3 ST (150K), and Magpie Pro Qwen2 ST (150K).
    • General OCR (8.9%, ~281K samples): OCR-VQA (80K), HME100K (74.5K), SynthDog-EN (40.1K), TextOCR-GPT4V (25.1K), TextCaps (21.9K), K12 Printing (12.8K), Rendered Text (10K), ChromeWriting (8.8K), IAM (5.7K), IIIT5K (2.0K).

    2. OneVision Multi-Modality Mixture (1.6M samples)

    • Multi-Image (43.0%, ~560K samples): NLVR2 (86K), Co-Instruct (50K), ScanNet (49.9K), RAVEN (35K), IconQA (34.6K), VIST (26K), ScanQA (25.6K), ContrastCaption (25.2K), ALFRED (22.6K), FlintstonesSV (22.3K), ImageCode (16.6K), DreamSim (15.9K), Birds-to-Words (14.3K), PororoSV (12.3K), Spot-the-Diff (10.8K), VISION (9.9K), nuScenes (9.8K), WebQA (9.3K), RecipeQA ImageCoherence (8.7K), RecipeQA VisualCloze (8.7K), TQA multi-image (8.2K), HQ-Edit-Diff (7.0K), AESOP (6.9K), MagicBrush-Diff (6.7K), COMICS-Dialogue (5.9K), Multi-VQA (5.0K), VizWiz multi-image (4.9K), CLEVR-Change (3.9K), NextQA multi-image (3.9K), IEdit (3.5K), Star (3.0K), DocVQA multi-image (1.9K), OCR-VQA multi-image (1.9K), MIT-PropertyCoherence (1.9K), MIT-StateCoherence (1.9K).
    • Single-Image Repetition (31.2%, ~800K samples): Sampled high-quality subsets including Magpie Pro (90K), Vision FLAN filtered (55.8K), Image Textualization (49.8K), Cauldron (40.2K), UReader (39.9K), Cambrian GPT-4o (24.9K), ShareGPT4V (21K), ALLaVA Inst (21K), LLaVA-Wild (10.9K), LAION-GPT4V (8.0K), LLaVA-158K (7.0K), Geo170K QA/Align (12.8K), ShareGPT4o (5.7K), TabMWP (4.5K), LLAVAR GPT4 (4.0K), MapQA (4.3K), MathQA (3.0K), TextOCR (2.5K), TextCaps (2.2K), ScienceQA (1.9K), FigureQA (1.8K), GeoQA+ (1.7K), and others.
    • Video (25.9%, ~350K samples): ShareGPT4Video (255K), YouCook2 (41.9K), Charades (23.6K), NextQA (9.5K), ActivityNet (6.5K), Ego4D (0.8K).
  6. Knowl 6 — LLaVA-OneVision Performance on Single-Image Benchmarks

    empirical result

    LLaVA-OneVision models (0.5B, 7B, 72B) were evaluated on single-image perception, OCR, mathematical reasoning, and visual chat benchmarks under zero-shot greedy decoding using the LMMs-Eval framework.

    Model AI2D ChartQA DocVQA (test) InfoVQA (test) MathVerse MathVista MMBench MMMU (val)
    Qwen-VL-Max 79.3 79.8 93.1 - 23.0 51.0 77.6 51.4
    Gemini-1.5-Pro 94.4 87.2 93.1 81.0 - 63.9 - 62.2
    Claude 3.5 Sonnet 94.7 90.8 95.2 49.7 - 67.7 - 68.3
    GPT-4V 78.2 78.5 88.4 - 32.8 49.9 75.0 56.8
    GPT-4o 94.2 85.7 92.8 - 50.2 63.8 - 69.1
    Cambrian-34B 79.7 73.8 75.5 - - 53.2 81.4 49.7
    InternVL-2-8B 83.8 83.3 91.6 74.8 27.5 58.3 81.7 49.3
    InternVL-2-26B 84.5 84.9 92.9 75.9 31.3 59.4 83.4 48.3
    LLaVA-OV-0.5B (SI) 54.2 61.0 71.2 41.3 17.3 34.6 43.8 31.2
    LLaVA-OV-0.5B 57.1 61.4 70.0 41.8 17.9 34.8 52.1 31.4
    LLaVA-OV-7B (SI) 81.6 78.8 86.9 65.3 26.9 56.1 81.7 47.3
    LLaVA-OV-7B 81.4 80.0 87.5 68.8 26.2 63.2 80.8 48.8
    LLaVA-OV-72B (SI) 85.1 84.9 91.8 74.6 37.7 66.5 86.6 57.4
    LLaVA-OV-72B 85.6 83.7 91.3 74.9 39.1 67.5 85.9 56.8
    Model MMVet MMStar SEED-Image ScienceQA RealWorldQA Vibe-Eval MM-LiveBench LLaVA-Wilder
    GPT-4V 49.9 57.1 49.9 75.7 61.4 57.9 - 81.0
    GPT-4o 76.2 - 76.2 - 58.6 63.1 92.4 85.9
    InternVL-2-8B 60.0 59.4 76.0 97.0 64.4 46.7 73.4 62.5
    InternVL-2-26B 65.4 60.4 76.8 97.5 66.8 51.5 77.2 70.2
    LLaVA-OV-0.5B 29.1 37.5 65.5 67.2 55.6 33.8 49.9 55.0
    LLaVA-OV-7B 57.5 61.7 75.4 96.0 66.3 51.7 77.1 67.8
    LLaVA-OV-72B 63.7 66.1 78.0 90.3 71.9 50.7 81.5 72.0

    LLaVA-OV-72B outperforms GPT-4V across multiple core benchmarks (e.g., AI2D: 85.6% vs 78.2%; ChartQA: 83.7% vs 78.5%; DocVQA: 91.3% vs 88.4%; MathVista: 67.5% vs 49.9%; MMBench: 85.9% vs 75.0%; RealWorldQA: 71.9% vs 61.4%) and achieves parity with GPT-4o on MathVista (67.5% vs 63.8%).

  7. Knowl 7 — LLaVA-OneVision Performance on Multi-Image and Multi-View Benchmarks

    empirical result

    LLaVA-OneVision models were evaluated across in-domain multi-image, in-domain 3D multi-view, and out-of-domain multi-image reasoning benchmarks (all results reported as accuracy %):

    Model In-Domain Multi-Image In-Domain 3D Multi-View Out-Domain
    IEI MI-VQA NLVR2 SDiff 3D-Chat ScanQA nuScenes BLINK Mantis MathVerse MuirBench
    GPT-4V 11.0 52.0 88.8 12.5 31.2 32.6 63.7 51.1 62.7 60.3 62.3
    LLaVA-N-Inter-7B 24.3 87.5 88.8 37.1 - - - 52.6 62.7 32.8 38.9
    Mantis-7B 11.2 52.5 87.4 17.6 2.6 16.1 46.2 46.4 59.5 27.2 36.1
    LLaVA-OV-0.5B (SI) 15.6 44.8 56.1 8.5 22.1 16.9 8.2 37.9 38.2 20.9 22.7
    LLaVA-OV-0.5B 17.1 48.7 63.4 36.4 60.0 29.4 70.5 52.1 39.6 60.0 25.5
    LLaVA-OV-7B (SI) 20.5 60.3 75.9 7.9 24.5 22.1 70.8 45.6 54.2 26.3 32.7
    LLaVA-OV-7B 22.2 90.2 89.4 39.2 62.8 30.1 79.8 48.2 64.2 67.6 41.8
    LLaVA-OV-72B (SI) 22.1 61.2 78.9 15.6 30.8 21.9 75.5 46.0 56.8 58.6 33.2
    LLaVA-OV-72B 22.5 95.3 93.8 43.3 63.2 35.8 78.8 55.4 77.6 91.6 54.8

    Key Observations

    1. Transfer from Single-Image Pretraining: The single-image trained checkpoint LLaVA-OV-72B (SI) achieves competitive performance on multi-image tasks (e.g., 61.2% on MI-VQA and 56.8% on Mantis) without seeing multi-image data, demonstrating representation transferability.
    2. Gains from OneVision Stage: Fine-tuning on mixed OneVision data dramatically improves multi-image reasoning: LLaVA-OV-72B surges on Spot-the-Difference (SDiff: 15.6% \rightarrow 43.3%), Multi-Image VQA (MI-VQA: 61.2% \rightarrow 95.3%), 3D Dialogue (3D-Chat: 30.8% \rightarrow 63.2%), and out-domain multi-image MathVerse (58.6% \rightarrow 91.6%), significantly outperforming GPT-4V (60.3%).
  8. Knowl 8 — LLaVA-OneVision Performance on Video Understanding Benchmarks

    empirical result

    LLaVA-OneVision was benchmarked across multi-choice and open-ended video benchmarks under 0-shot settings:

    Model ActNet-QA EgoSchema MLVU MVBench NextQA PercepTest SeedBench (video) VideoMME (wo/w subs)
    GPT-4V 57.0 - 49.2 43.5 - - 60.5 59.9 / 63.3
    GPT-4o - - 64.6 - - - - 71.9 / 77.2
    Gemini-1.5-Pro 57.5 72.2 - - - - - 75.0 / 81.3
    VILA-40B 58.0 58.0 - - 67.9 54.0 - 60.1 / 61.1
    LLaVA-N-Video-32B 54.3 60.9 65.5 - 77.3 59.4 - 60.2 / 63.0
    LLaVA-OV-0.5B (SI) 49.0 33.1 47.9 43.3 53.6 48.6 43.4 41.7 / 40.4
    LLaVA-OV-0.5B 50.5 26.8 50.3 45.5 57.2 49.2 44.2 44.0 / 43.5
    LLaVA-OV-7B (SI) 55.1 52.9 60.2 51.2 61.6 54.9 51.1 55.0 / 59.1
    LLaVA-OV-7B 56.6 60.1 64.7 56.7 79.4 57.1 56.9 58.2 / 61.5
    LLaVA-OV-72B (SI) 62.1 58.6 60.9 57.1 67.2 62.3 60.9 64.8 / 66.9
    LLaVA-OV-72B 62.3 62.0 68.0 59.4 80.2 66.9 62.1 66.2 / 69.5

    Quantitative Highlights

    • LLaVA-OV-72B achieves 62.3% on ActivityNet-QA, 62.0% on EgoSchema, 68.0% on MLVU, 80.2% on NextQA, 66.9% on PerceptionTest, and 66.2% on VideoMME (without subtitles), outperforming previous open video LMMs and competitive with GPT-4V.
    • Zero-Shot Modality Transfer: Single-image-only models (SI) exhibit strong zero-shot video transfer. For example, LLaVA-OV-72B (SI) achieves 62.1% on ActivityNet-QA (vs 62.3% after video fine-tuning) and 64.8% on VideoMME (vs 66.2% after video fine-tuning).
  9. Knowl 9 — Emergent Cross-Scenario Capabilities via Visual Task Transfer

    model/method

    Joint training across single-image, multi-image, and video representations induces emergent zero-shot compositional capabilities across scenarios:

    1. S1: Joint Diagram and Chart Reasoning: Successfully correlates information across separate diagram and chart images within a multi-image prompt, despite never observing joint diagram-plus-chart multi-image pairs during training.
    2. S2: Multi-Image GUI Navigation for Agents: Combines fine-grained single-image OCR with multi-image sequential reasoning to interpret chronological smartphone UI screens and output multi-step tap action instructions to achieve user goals.
    3. S3: Set-of-Mark (SoM) Prompting: Emergent capability to accurately refer to and reason about fine-grained visual elements tagged with arbitrary numerical overlay markers in images, composed from visual referring and OCR without explicit SoM training.
    4. S4: Image-to-Video Editing Instruction: Composes single-image edit instruction following and video captioning to generate coherent step-by-step video creation instructions that transform a static image into a dynamic video scene.
    5. S5: Video-to-Video Difference Analysis: Generalizes multi-image difference detection (spot-the-difference) to video pairs, analyzing temporal and entity discrepancies between two video clips that share common starting frames or similar backgrounds.
    6. S6: Multi-Camera Autonomous Driving Understanding: Synthesizes multi-panel image understanding, video narration, and spatio-temporal reasoning to interpret synchronized 4-camera feeds from autonomous vehicles and predict ego-vehicle trajectories.
    7. S7: Composed Sub-Video Analysis: Deconstructs vertical videos containing distinct split sub-scenes (e.g., reacting user in the foreground and a video clip in the background) into layered narrative descriptions.
    8. S8: Visual Prompting in Video: Detects and reads visual cues (e.g., semi-transparent circular bounding highlights) directly in video frames without video-specific visual prompt annotations.
    9. S9: Cross-Modality Visual Referring: Identifies whether a subject from an isolated static reference image appears within an arbitrary video clip, demonstrating cross-scenario entity linking.
  10. Knowl 10 — Asymmetric Scaling Dynamics of Perception and Reasoning in LMMs

    empirical result

    Empirical comparisons between LLaVA-OneVision model scales (0.5B, 7B, 72B) and training stages (Single-Image vs. OneVision) reveal distinct scaling dynamics for perceptual versus reasoning tasks:

    1. Vision-Encoder Dominance in Pure Perception: On perception-heavy tasks like PerceptionTest, scaling the LLM backbone from 0.5B to 7B yields only a modest +0.6 point improvement (48.6% \rightarrow 49.2% for 0.5B to 54.9% \rightarrow 57.1% for 7B), whereas other video benchmarks improve by >5>5 points. This demonstrates that basic visual perception capabilities are primarily constrained by vision encoder representation rather than language model parameter scale.
    2. LLM Dominance in Long-Context Reasoning: For reasoning-heavy long-video tasks like EgoSchema, performance scales dramatically with LLM capacity: LLaVA-OV-0.5B scores 26.8%, LLaVA-OV-7B achieves 60.1% (+33.3 points), and LLaVA-OV-72B reaches 62.0%.
    3. Static-Frame Question Answering in Video Benchmarks: On ActivityNet-QA, the single-image trained model LLaVA-OV-7B (SI) scores 55.1%, which is nearly identical to the 56.6% obtained after mixed video fine-tuning LLaVA-OV-7B. Analysis indicates that when questions can be answered via static cues visible across all frames (e.g., "What's the color of the ball?"), single-image pretraining transfers directly to video QA with minimal domain gap.

Coverage note — Omitted the high-level historical overview of the LLaVA-NeXT blog series from January to June 2024 (Section 1 and Appendix A) and standard open-source release notices (Section 1 and Appendix E.4), as they represent development history and logistics rather than technical contributions.

References

  1. 1.Manoj Acharya, Kushal Kafle, and Christopher Kanan. Tallyqa: Answering complex counting questions. In AAAI, 2019. 39
  2. 2.Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation-based formalisms, 2019. 39
  3. 3.Anthropic. Claude-3.5. https://www.anthropic.com/news/claude-3-5-sonnet, 2024. 2, 11
  4. 4.Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In ICCV, 2015. 39
  5. 5.Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19129–19139, 2022. 9
  6. 6.Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 40
  7. 7.Haoping Bai, Shancong Mou, Tatiana Likhomanenko, Ramazan Gokberk Cinbis, Oncel Tuzel, Ping Huang, Jiulong Shan, Jianjun Shi, and Meng Cao. Vision datasets: A benchmark for vision-based industrial inspection, 2023. 40
  8. 8.Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. Technical Report, 2023. 11, 37
  9. 9.Ankan Bansal, Yuting Zhang, and Rama Chellappa. Visual question answering on image sets. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI 16, pages 51–67. Springer, 2020. 9
  10. 10.Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07726, 2024. 2
  11. 11.Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Marçal Rusinol, Ernest Valveny, CV Jawahar, and Dimosthenis Karatzas. Scene text visual question answering. In ICCV, 2019. 39
  12. 12.Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving, 2020. 40
  13. 13.Jimmy Carter. Textocr-gpt4v. https://huggingface.co/datasets/jimmycarter/textocr-gpt4v, 2024. 39
  14. 14.Shuaichen Chang, David Palzer, Jialin Li, Eric Fosler-Lussier, and Ningchuan Xiao. Mapqa: A dataset for question answering on choropleth maps, 2022. 39
  15. 15.Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, and Yonatan Bisk. Webqa: Multihop and multimodal qa. arXiv preprint arXiv:2109.00590, 2021. 40
  16. 16.Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. Allava: Harnessing gpt4v-synthesized data for a lite vision-language model. arXiv preprint arXiv:2402.11684, 2024. 6, 7, 39
  17. 17.Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen, and Xiaodan Liang. Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression, 2022. 39
  18. 18.Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P. Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning, 2022. 39
  19. 19.Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024. 10
  20. 20.Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12793, 2023. 5
  21. 21.Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, Li Yuan, Yu Qiao, Dahua Lin, Feng Zhao, and Jiaqi Wang. Sharegpt4video: Improving video understanding and generation with better captions. arXiv preprint arXiv:2406.04325, 2024. 38, 40
  22. 22.Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238, 2023. 9, 11, 37, 39
  23. 23.Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. Hitab: A hierarchical table dataset for question answering and natural language generation. In ACL, 2022. 39
  24. 24.Yew Ken Chia, Vernon Toh Yan Han, Deepanway Ghosal, Lidong Bing, and Soujanya Poria. Puzzlevqa: Diagnosing multimodal reasoning challenges of language models with abstract visual patterns. arXiv preprint arXiv:2403.13315, 2024. 9
  25. 25.Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017. 40
  26. 26.Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. In NeurIPS, 2024. 2
  27. 27.Maxwell Forbes, Christine Kaeser-Chen, Piyush Sharma, and Serge Belongie. Neural naturalist: Generating fine-grained image comparisons, 2019. 40
  28. 28.Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models, 2024. 10, 36, 38
  29. 29.Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075, 2024. 10, 11
  30. 30.Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data, 2023. 40
  31. 31.Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. arXiv preprint arXiv:2404.12390, 2024. 9, 10
  32. 32.Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, and Lingpeng Kong. G-llava: Solving geometric problem with multi-modal large language model, 2023. 39
  33. 33.Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhongcong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Cartillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli, Christoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Abrham Gebreselasie, Cristina Gonzalez, James Hillis, Xuhua Huang, Yifei Huang, Wenqi Jia, Weslie Khoo, Jachym Kolar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li, Yanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Modhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will Price, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran Somasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao, Minh Vo, Yuchen Wang, Xindi Wu, Takuma Yagi, Ziwei Zhao, Yunyi Zhu, Pablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria Farinella, Christian Fuegen, Bernard Ghanem, Vamsi Krishna Ithapu, C. V. Jawahar, Hanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude Oliva, Hyun Soo Park, James M. Rehg, Yoichi Sato, Jianbo Shi, Mike Zheng Shou, Antonio Torralba, Lorenzo Torresani, Mingfei Yan, and Jitendra Malik. Ego4d: Around the world in 3,000 hours of egocentric video, 2022. 38, 40
  34. 34.Ziyu Guo, Renrui Zhang, Hao Chen, Jialin Gao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Sciverse. https://sciverse-cuhk.github.io, 2024. 9
  35. 35.Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li, Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understanding, generation, and instruction following. arXiv preprint arXiv:2309.00615, 2023. 2
  36. 36.Tanmay Gupta, Dustin Schwenk, Ali Farhadi, Derek Hoiem, and Aniruddha Kembhavi. Imagine this! scripts to compositions to videos, 2018. 40
  37. 37.Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In CVPR, 2018. 39, 40
  38. 38.Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 36:20482–20494, 2023. 9
  39. 39.Mehrdad Hosseinzadeh and Yang Wang. Image change captioning by learning from an auxiliary task. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2724–2733, 2021. 40
  40. 40.Ting-Hao K. Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin, Aishwarya Agrawal, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, et al. Visual storytelling. In 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2016), 2016. 9
  41. 41.Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR, 2019. 39
  42. 42.Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang, Yuyin Zhou, and Cihang Xie. Hq-edit: A high-quality dataset for instruction-based image editing, 2024. 40
  43. 43.Phillip Isola, Joseph J. Lim, and Edward H. Adelson. Discovering states and transformations in image collections. In CVPR, 2015. 40
  44. 44.Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan Boyd-Graber, Hal Daumé III au2, and Larry Davis. The amazing mysteries of the gutter: Drawing inferences between panels in comic book narratives, 2017. 40
  45. 45.Harsh Jhamtani and Taylor Berg-Kirkpatrick. Learning to describe differences between pairs of similar images. arXiv preprint arXiv:1808.10584, 2018. 9
  46. 46.Harsh Jhamtani and Taylor Berg-Kirkpatrick. Learning to describe differences between pairs of similar images, 2018. 40
  47. 47.Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. arXiv preprint arXiv:2405.01483, 2024. 2, 10, 12, 40
  48. 48.Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In CVPR, 2017. 39
  49. 49.Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. Dvqa: Understanding data visualizations via question answering. In CVPR, 2018. 37, 39
  50. 50.Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Akos Kadar, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning, 2018. 39
  51. 51.Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. Prismatic vlms: Investigating the design space of visually-conditioned language models. Technical Report, 2024. 2
  52. 52.Mehran Kazemi, Hamidreza Alvari, Ankit Anand, Jialin Wu, Xi Chen, and Radu Soricut. Geomverse: A systematic evaluation of large models for geometric reasoning. arXiv preprint arXiv:2312.12241, 2023. 39
  53. 53.Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In ECCV, 2016. 10, 37, 39
  54. 54.Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 235–251. Springer, 2016. 9, 36, 38
  55. 55.Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension. In Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, pages 4999–5007, 2017. 39
  56. 56.Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5376–5384, 2017. 40
  57. 57.Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. The hateful memes challenge: Detecting hate speech in multimodal memes. In NeurIPS, 2020. 39
  58. 58.Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Ocr-free document understanding transformer. In European Conference on Computer Vision (ECCV), 2022. 37, 39
  59. 59.Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Fei-Fei Li. Visual genome: Connecting language and vision using crowdsourced dense image annotations, 2016. 39
  60. 60.Benno Krojer, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, and Siva Reddy. Image retrieval from contextual descriptions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Online, May 2022. Association for Computational Linguistics. 40
  61. 61.Shanghai AI Laboratory. Sharegpt-4o: Comprehensive multimodal annotations with gpt-4o, 2023. 38
  62. 62.Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1–10, 2018. 39
  63. 63.Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? Technical Report, 2024. 2, 6, 37
  64. 64.Bo Li, Hao Zhang, Kaichen Zhang, Dong Guo, Yuanhan Zhang, Renrui Zhang, Feng Li, Ziwei Liu, and Chunyuan Li. Llava-next: What else influences visual instruction tuning beyond data?, May 2024. 1, 2, 3, 5, 34, 35
  65. 65.Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. Llava-next: Stronger llms supercharge multimodal capabilities in the wild, May 2024. 1, 3, 9, 10, 34, 36, 38
  66. 66.Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal llms with generative comprehension, 2023. 10
  67. 67.Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao, et al. Multimodal foundation models: From specialists to general-purpose assistants. Foundations and Trends® in Computer Graphics and Vision, 2024. 1
  68. 68.Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next: Tackling multi-image, video, and 3d in large multimodal models, June 2024. 1, 2, 5, 6, 7, 9, 10, 12, 34, 35, 36, 38
  69. 69.Juncheng Li, Kaihang Pan, Zhiqi Ge, Minghe Gao, Wei Ji, Wenqiao Zhang, Tat-Seng Chua, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. Fine-tuning multimodal llms to follow zero-shot demonstrative instructions, 2024. 7, 40
  70. 70.Juncheng Li, Kaihang Pan, Zhiqi Ge, Minghe Gao, Hanwang Zhang, Wei Ji, Wenqiao Zhang, Tat-Seng Chua, Siliang Tang, and Yueting Zhuang. Empowering vision-language models to follow interleaved vision-language instructions. arXiv preprint arXiv:2308.04152, 2023. 2, 12
  71. 71.Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. Mvbench: A comprehensive multi-modal video understanding benchmark, 2023. 10
  72. 72.Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision, 2024. 2
  73. 73.Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia. Mini-gemini: Mining the potential of multi-modality vision language models. Technical Report, 2024. 2
  74. 74.Yitong Li, Zhe Gan, Yelong Shen, Jingjing Liu, Yu Cheng, Yuexin Wu, Lawrence Carin, David Carlson, and Jianfeng Gao. Storygan: A sequential conditional gan for story visualization, 2019. 40
  75. 75.Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan Yuille. Super-clevr: A virtual benchmark to diagnose domain robustness in visual reasoning, 2023. 39
  76. 76.Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122, 2023. 2
  77. 77.Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26689–26699, 2024. 2, 11, 12
  78. 78.Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context, 2015. 37, 39
  79. 79.Fangyu Liu, Guy Edward Toh Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 2023. 39
  80. 80.Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv:2306.14565, 2023. 39
  81. 81.Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, 2024. 1, 3, 6, 37
  82. 82.Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. 1, 4, 5, 12, 34, 37
  83. 83.Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 1, 2, 3, 6, 39
  84. 84.Xuejing Liu, Wei Tang, Xinzhe Ni, Jinghui Lu, Rui Zhao, Zechao Li, and Fei Tan. What large language models bring to text-rich vqa?, 2023. 10
  85. 85.Xuejing Liu, Wei Tang, Xinzhe Ni, Jinghui Lu, Rui Zhao, Zechao Li, and Fei Tan. What large language models bring to text-rich vqa? arXiv preprint arXiv:2311.07306, 2023. 9
  86. 86.Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, and Ziwei Liu. Mmbench: Is your multi-modal model an all-around player? Technical Report, 2023. 9, 10, 36
  87. 87.LMMs-Lab. Video detail caption, 2024. 11
  88. 88.Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. In International Conference on Machine Learning, pages 22631–22648. PMLR, 2023. 2
  89. 89.Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, and Yaofeng Sun. Deepseek-vl: towards real-world vision-language understanding. Technical Report, 2024. 37
  90. 90.Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating math reasoning in visual contexts with gpt-4v, bard, and other large multimodal models. arXiv preprint arXiv:2310.02255, 2023. 9, 10, 36, 38
  91. 91.Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning, 2021. 39
  92. 92.Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. In ACL, 2021. 39
  93. 93.Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In The 36th Conference on Neural Information Processing Systems (NeurIPS), 2022. 10, 39
  94. 94.Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. In International Conference on Learning Representations (ICLR), 2023. 39
  95. 95.Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning. In NeurIPS, 2021. 39, 40
  96. 96.Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424, 2023. 11
  97. 97.Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 2024. 10
  98. 98.Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic benchmark for very long-form video language understanding. Advances in Neural Information Processing Systems, 36, 2024. 10, 11
  99. 99.Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In CVPR, 2019. 39
  100. 100.U-V Marti and Horst Bunke. The iam-database: an english sentence database for offline handwriting recognition. International journal on document analysis and recognition, 5:39–46, 2002. 39
  101. 101.Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In ACL, 2022. 9, 10, 36, 37, 39
  102. 102.Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1697–1706, 2022. 9, 10, 36, 39
  103. 103.Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In WACV, 2021. 9, 10, 36, 37, 39, 40
  104. 104.Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, Bowen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Floris Weers, et al. Mm1: Methods, analysis & insights from multimodal llm pre-training. arXiv preprint arXiv:2403.09611, 2024. 2
  105. 105.A. Mishra, K. Alahari, and C. V. Jawahar. Scene text recognition using higher order language priors. In BMVC, 2012. 39
  106. 106.Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In ICDAR, 2019. 39
  107. 107.Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 947–952, 2019. 40
  108. 108.Jason Obeid and Enamul Hoque. Chart-to-text: Generating natural language descriptions for charts by adapting the transformer model, 2020. 39
  109. 109.OpenAI. Gpt-4v. https://openai.com/index/gpt-4v-system-card/, 2023. 2, 9, 11, 12
  110. 110.OpenAI. Hello gpt-4o. https://openai.com/index/hello-gpt-4o/, 2024. 2, 9, 11, 12
  111. 111.Piotr Padlewski, Max Bain, Matthew Henderson, Zhongkai Zhu, Nishant Relan, Hai Pham, Donovan Ong, Kaloyan Aleksiev, Aitor Ormazabal, Samuel Phua, et al. Vibe-eval: A hard evaluation suite for measuring progress of multimodal language models. arXiv preprint arXiv:2405.02287, 2024. 9
  112. 112.Piotr Padlewski, Max Bain, Matthew Henderson, Zhongkai Zhu, Nishant Relan, Hai Pham, Donovan Ong, Kaloyan Aleksiev, Aitor Ormazabal, Samuel Phua, Ethan Yeo, Eugenie Lamprecht, Qi Liu, Yuqi Wang, Eric Chen, Deyu Fu, Lei Li, Che Zheng, Cyprien de Masson d’Autume, Dani Yogatama, Mikel Artetxe, and Yi Tay. Vibe-eval: A hard evaluation suite for measuring progress of multimodal language models, 2024. 10, 36, 38
  113. 113.Dong Huk Park, Trevor Darrell, and Anna Rohrbach. Robust change captioning, 2019. 40
  114. 114.Renjie Pi, Jianshu Zhang, Jipeng Zhang, Rui Pan, Zhekai Chen, and Tong Zhang. Image textualization: An automatic framework for creating accurate and detailed image descriptions, 2024. 39
  115. 115.Viorica Patr ˘ aucean, Lucas Smaira, Ankush Gupta, Adrià Recasens Continente, Larisa Markeeva, Dylan Banarse, Skanda Koppula, Joseph Heyward, Mateusz Malinowski, Yi Yang, Carl Doersch, Tatiana Matejovicova, Yury Sulsky, Antoine Miech, Alex Frechette, Hanna Klimczak, Raphael Koster, Junlin Zhang, Stephanie Winkler, Yusuf Aytar, Simon Osindero, Dima Damen, Andrew Zisserman, and João Carreira. Perception test: A diagnostic benchmark for multimodal video models. In Advances in Neural Information Processing Systems, 2023. 10, 11
  116. 116.Yuxuan Qiao, Haodong Duan, Xinyu Fang, Junming Yang, Lin Chen, Songyang Zhang, Jiaqi Wang, Dahua Lin, and Kai Chen. Prism: A framework for decoupling and assessing the capabilities of vlms, 2024. 12
  117. 117.Harsh Raj, Janhavi Dadhania, Akhilesh Bhardwaj, and Prabuchandran KJ. Multi-image visual question answering. arXiv preprint arXiv:2112.13706, 2021. 9
  118. 118.Hareesh Ravi, Kushal Kafle, Scott Cohen, Jonathan Brandt, and Mubbasir Kapadia. Aesop: Abstract encoding of stories, objects, and pictures. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 2032–2043, 2021. 40
  119. 119.Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-okvqa: A benchmark for visual question answering using world knowledge. In ECCV, 2022. 39
  120. 120.Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. Solving geometry problems: Combining text and diagram interpretation. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1466–1476, 2015. 39
  121. 121.ShareGPT. https://sharegpt.com/, 2023. 37, 39
  122. 122.Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10740–10749, 2020. 9, 40
  123. 123.Oleksii Sidorov, Ronghang Hu, Marcus Rohrbach, and Amanpreet Singh. Textcaps: a dataset for image captioning with reading comprehension, 2020. 39
  124. 124.Gunnar A. Sigurdsson, Gül Varol, Xiaolong Wang, Ivan Laptev, Ali Farhadi, and Abhinav Gupta. Hollywood in homes: Crowdsourcing data collection for activity understanding. ArXiv e-prints, 2016. 38, 40
  125. 125.Alane Suhr, Mike Lewis, James Yeh, and Yoav Artzi. A corpus of natural language for visual reasoning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 217–223, 2017. 9
  126. 126.Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. A corpus for reasoning about natural language grounded in photographs, 2019. 40
  127. 127.Hao Tan, Franck Dernoncourt, Zhe Lin, Trung Bui, and Mohit Bansal. Expressing visual relationships via language, 2019. 40
  128. 128.Ryota Tanaka, Kyosuke Nishida, and Sen Yoshida. Visualmrc: Machine reading comprehension on document images. In AAAI, 2021. 39
  129. 129.Benny J. Tang, Angie Boggust, and Arvind Satyanarayan. Vistext: A benchmark for semantically rich chart captioning, 2023. 39
  130. 130.Gemini Team. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024. 11
  131. 131.Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 2, 12
  132. 132.Ting-Hao, Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell. Visual storytelling, 2016. 40
  133. 133.Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. arXiv preprint arXiv:2406.16860, 2024. 2, 6, 9, 11, 39
  134. 134.Bryan Wang, Gang Li, Xin Zhou, Zhourong Chen, Tovi Grossman, and Yang Li. Screen2words: Automatic mobile ui summarization with multimodal learning, 2021. 36, 39
  135. 135.Fei Wang, Xingyu Fu, James Y Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. Muirbench: A comprehensive benchmark for robust multi-image understanding. arXiv preprint arXiv:2406.09411, 2024. 9, 10
  136. 136.Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021. 2
  137. 137.Chris Wendler. wendlerc/renderedtext, 2023. 39
  138. 138.Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-language understanding, 2024. 10
  139. 139.Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, et al. Q-bench: A benchmark for general-purpose foundation models on low-level vision. arXiv preprint arXiv:2309.14181, 2023. 9
  140. 140.Haoning Wu, Hanwei Zhu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Annan Wang, Wenxiu Sun, Qiong Yan, Xiaohong Liu, Guangtao Zhai, Shiqi Wang, and Weisi Lin. Towards open-ended visual quality comparison, 2024. 40
  141. 141.x.ai. Grok-1.5 vision preview. 9, 10
  142. 142.Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question-answering to explaining temporal actions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9777–9786, June 2021. 11, 38, 40
  143. 143.Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng. Pllava: Parameter-free llava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404.16994, 2024. 12
  144. 144.Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing. ArXiv, abs/2406.08464, 2024. 36, 37, 39
  145. 145.Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, and Lifu Huang. Vision-flan: Scaling human-labeled tasks in visual instruction tuning. arXiv preprint arXiv:2402.11690, 2024. 2
  146. 146.Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, and Lifu Huang. Vision-flan: Scaling human-labeled tasks in visual instruction tuning, 2024. 37, 39
  147. 147.Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Ikizler-Cinbis. Recipeqa: A challenge dataset for multimodal comprehension of cooking recipes, 2018. 40
  148. 148.An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024. 3, 35
  149. 149.Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441, 2023. 13
  150. 150.Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, Qin Jin, Liang He, Xin Alex Lin, and Fei Huang. Ureader: Universal ocr-free visually-situated language understanding with multimodal large language model, 2023. 37, 39
  151. 151.Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. arXiv preprint arXiv:2306.13549, 2023. 9
  152. 152.Licheng Yu, Patrick Poirson, Shan Yang, Alexander C. Berg, and Tamara L. Berg. Modeling context in referring expressions, 2016. 39
  153. 153.Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities, 2023. 10
  154. 154.Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490, 2023. 9
  155. 155.Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. Activitynet-qa: A dataset for understanding complex web videos via question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 9127–9134, 2019. 10, 11, 38, 40
  156. 156.Ye Yuan, Xiao Liu, Wondimu Dikubab, Hui Liu, Zhilong Ji, Zhongqin Wu, and Xiang Bai. Syntax-aware network for handwritten mathematical expression recognition. arXiv preprint arXiv:2203.01601, 2022. 39
  157. 157.Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, and Yuxuan Sun. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In CVPR, 2024. 9, 10, 36, 38
  158. 158.Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975–11986, 2023. 3, 35
  159. 159.Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, and Song-Chun Zhu. Raven: A dataset for relational and analogical visual reasoning. In CVPR, 2019. 39, 40
  160. 160.Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guided image editing, 2024. 40
  161. 161.Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint arXiv:2407.12772, 2024. 8, 9, 10, 36
  162. 162.Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, et al. Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual input and output. arXiv preprint arXiv:2407.03320, 2024. 2, 11, 12
  163. 163.Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint arXiv:2406.16852, 2024. 12
  164. 164.Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199, 2023. 2
  165. 165.Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? arXiv preprint arXiv:2403.14624, 2024. 9, 10
  166. 166.Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Yichi Zhang, Ziyu Guo, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, Shanghang Zhang, Peng Gao, and Hongsheng Li. Mavis: Mathematical visual instruction tuning, 2024. 39
  167. 167.Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander Hauptmann, Yonatan Bisk, et al. Direct preference optimization of video large multimodal models from language model reward. arXiv preprint arXiv:2404.01258, 2024. 38
  168. 168.Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, and Tong Sun. Llavar: Enhanced visual instruction tuning for text-rich image understanding. arXiv preprint arXiv:2306.17107, 2023. 39
  169. 169.Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, April 2024. 1, 5, 6, 12, 34, 35, 36
  170. 170.Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264, 2024. 10, 11
  171. 171.Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding, 2024. 36
  172. 172.Luowei Zhou, Chenliang Xu, and Jason J. Corso. Towards automatic learning of procedures from web instructional videos, 2017. 38, 40
  173. 173.Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 2
  174. 174.Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei-Fei. Visual7w: Grounded question answering in images. In CVPR, 2016. 39

Citation

MLA
Li, B., et al. “LLaVA-OneVision: Easy Visual Task Transfer”. arXiv, 2024, https://doi.org/10.48550/arXiv.2408.03326.
APA
Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., & Li, C. (2024). LLaVA-OneVision: Easy Visual Task Transfer. arXiv. https://doi.org/10.48550/arXiv.2408.03326
Chicago
Li, B., Y. Zhang, D. Guo, et al. 2024. “LLaVA-OneVision: Easy Visual Task Transfer”. Preprint, ArXiv. https://doi.org/10.48550/arXiv.2408.03326.
Harvard
Li, B. et al. (2024) “LLaVA-OneVision: Easy Visual Task Transfer”. arXiv. Available at: https://doi.org/10.48550/arXiv.2408.03326.
Vancouver
1. Li B, Zhang Y, Guo D, et al (2024) LLaVA-OneVision: Easy Visual Task Transfer. https://doi.org/10.48550/arXiv.2408.03326

BibTeX

@misc{https://doi.org/10.48550/arxiv.2408.03326,
  doi = {10.48550/ARXIV.2408.03326},
  url = {https://arxiv.org/abs/2408.03326},
  author = {Li, Bo and Zhang, Yuanhan and Guo, Dong and Zhang, Renrui and Li, Feng and Zhang, Hao and Zhang, Kaichen and Zhang, Peiyuan and Li, Yanwei and Liu, Ziwei and Li, Chunyuan},
  keywords = {Computer Vision and Pattern Recognition (cs.CV), Artificial Intelligence (cs.AI), Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {LLaVA-OneVision: Easy Visual Task Transfer},
  publisher = {arXiv},
  year = {2024},
  copyright = {Creative Commons Attribution Non Commercial No Derivatives 4.0 International}
}
Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF