title: LLaVA-OneVision: Easy Visual Task Transfer
summary: We present LLaVA-OneVision, a family of open large multimodal models (LMMs)
  developed by consolidating our insights into data, models, and visual
  representations in the LLaVA-NeXT blog series. Our experimental results
  demonstrate that LLaVA-OneVision is the first single model that can
  simultaneously push the performance boundaries of open LMMs in three important
  computer vision scenarios: single-image, multi-image, and video scenarios.
  Importantly, the design of LLaVA-OneVision allows strong transfer learning
  across different modalities/scenarios, yielding new emerging capabilities. In
  particular, strong video understanding and cross-scenario capabilities are
  demonstrated through task transfer from images to videos.
paper_authors._id: 66b2deea8188b0d6f73f4d95
paper_authors.name: Bo Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-09T07:48:21.928Z
paper_authors._id: 62d3f7d84b0933c48f3cdd9c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62d3f7d84b0933c48f3cdd9c/rSoSyH0Td9fD9HVlyK7bh.jpeg
paper_authors.fullname: Bo Li
paper_authors.type: user
paper_authors.user: luodian
paper_authors._id: 66b2deea8188b0d6f73f4d96
paper_authors.name: Yuanhan Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-08T10:58:10.922Z
paper_authors._id: 62a993d80472c0b7f94027df
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62a993d80472c0b7f94027df/j5vp-IwLA2YBexylUHiQU.png
paper_authors.fullname: Zhang Yuanhan
paper_authors.type: user
paper_authors.user: ZhangYuanhan
paper_authors._id: 66b2deea8188b0d6f73f4d97
paper_authors.name: Dong Guo
paper_authors._id: 66b2deea8188b0d6f73f4d98
paper_authors.name: Renrui Zhang
paper_authors._id: 66b2deea8188b0d6f73f4d99
paper_authors.name: Feng Li
paper_authors._id: 66b2deea8188b0d6f73f4d9a
paper_authors.name: Hao Zhang
paper_authors._id: 66b2deea8188b0d6f73f4d9b
paper_authors.name: Kaichen Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-08T07:40:31.239Z
paper_authors._id: 64bb77e786e7fb5b8a317a43
paper_authors.avatarUrl: /avatars/4c1720db323082e644babeb827c6f5d4.svg
paper_authors.fullname: kcz
paper_authors.type: user
paper_authors.user: kcz358
paper_authors._id: 66b2deea8188b0d6f73f4d9c
paper_authors.name: Yanwei Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-08T10:59:21.608Z
paper_authors._id: 643ff78cdc984afcbbbc3b1a
paper_authors.avatarUrl: /avatars/eec5198ce88aaf8156840bec0d190a7f.svg
paper_authors.fullname: Yanwei Li
paper_authors.type: user
paper_authors.user: YanweiLi
paper_authors._id: 66b2deea8188b0d6f73f4d9d
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-08T10:59:13.559Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_authors._id: 66b2deea8188b0d6f73f4d9e
paper_authors.name: Chunyuan Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-08T10:59:02.955Z
paper_authors._id: 62aba526cae4462c0c6caa0f
paper_authors.avatarUrl: /avatars/430560ec2c2547f819225769ab432f30.svg
paper_authors.fullname: Chunyuan Li
paper_authors.type: user
paper_authors.user: Chunyuan24
paper_title: LLaVA-OneVision: Easy Visual Task Transfer
paper_summary: We present LLaVA-OneVision, a family of open large multimodal models (LMMs)
  developed by consolidating our insights into data, models, and visual
  representations in the LLaVA-NeXT blog series. Our experimental results
  demonstrate that LLaVA-OneVision is the first single model that can
  simultaneously push the performance boundaries of open LMMs in three important
  computer vision scenarios: single-image, multi-image, and video scenarios.
  Importantly, the design of LLaVA-OneVision allows strong transfer learning
  across different modalities/scenarios, yielding new emerging capabilities. In
  particular, strong video understanding and cross-scenario capabilities are
  demonstrated through task transfer from images to videos.
paper_ai_keywords: multimodal models LLaVA-OneVision LLaVA-NeXT single-image multi-image video scenarios transfer learning task transfer video understanding cross-modal transfer