title: How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal
    Models with Open-Source Suites
summary: In this report, we introduce InternVL 1.5, an open-source multimodal large
  language model (MLLM) to bridge the capability gap between open-source and
  proprietary commercial models in multimodal understanding. We introduce three
  simple improvements: (1) Strong Vision Encoder: we explored a continuous
  learning strategy for the large-scale vision foundation model -- InternViT-6B,
  boosting its visual understanding capabilities, and making it can be
  transferred and reused in different LLMs. (2) Dynamic High-Resolution: we
  divide images into tiles ranging from 1 to 40 of 448times448 pixels
  according to the aspect ratio and resolution of the input images, which
  supports up to 4K resolution input. (3) High-Quality Bilingual Dataset: we
  carefully collected a high-quality bilingual dataset that covers common scenes,
  document images, and annotated them with English and Chinese question-answer
  pairs, significantly enhancing performance in OCR- and Chinese-related tasks.
  We evaluate InternVL 1.5 through a series of benchmarks and comparative
  studies. Compared to both open-source and proprietary models, InternVL 1.5
  shows competitive performance, achieving state-of-the-art results in 8 of 18
  benchmarks. Code has been released at https://github.com/OpenGVLab/InternVL.
paper_authors._id: 662b161666c46de6b3519b14
paper_authors.name: Zhe Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-09T14:49:44.356Z
paper_authors._id: 64119264f0f81eb569e0d569
paper_authors.avatarUrl: /avatars/00dccedea51ac7d2f398ca9d65a8e78b.svg
paper_authors.fullname: Zhe Chen
paper_authors.type: user
paper_authors.user: czczup
paper_authors._id: 662b161666c46de6b3519b15
paper_authors.name: Weiyun Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T07:49:56.737Z
paper_authors._id: 619507e7b74b6c591f794340
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
paper_authors.fullname: Weiyun Wang
paper_authors.type: user
paper_authors.user: Weiyun1025
paper_authors._id: 662b161666c46de6b3519b16
paper_authors.name: Hao Tian
paper_authors._id: 662b161666c46de6b3519b17
paper_authors.name: Shenglong Ye
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T07:52:57.771Z
paper_authors._id: 64804866c7f87934d082bb25
paper_authors.avatarUrl: /avatars/41761226c79ac16e48d4c4cb84362adb.svg
paper_authors.fullname: Yeshenglong
paper_authors.type: user
paper_authors.user: Yeshenglong
paper_authors._id: 662b161666c46de6b3519b18
paper_authors.name: Zhangwei Gao
paper_authors._id: 662b161666c46de6b3519b19
paper_authors.name: Erfei Cui
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T07:53:24.828Z
paper_authors._id: 6579b818563044badca392fc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6579b818563044badca392fc/XTKQ9Lhceibp9dnQADPQF.jpeg
paper_authors.fullname: cuierfei
paper_authors.type: user
paper_authors.user: cuierfei
paper_authors._id: 662b161666c46de6b3519b1a
paper_authors.name: Wenwen Tong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T07:53:31.787Z
paper_authors._id: 647d4f1236e109abce409c3b
paper_authors.avatarUrl: /avatars/d166f5f8be666e96b522a0a0effd21c4.svg
paper_authors.fullname: Wenwen Tong
paper_authors.type: user
paper_authors.user: tongww
paper_authors._id: 662b161666c46de6b3519b1b
paper_authors.name: Kongzhi Hu
paper_authors._id: 662b161666c46de6b3519b1c
paper_authors.name: Jiapeng Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T07:53:46.130Z
paper_authors._id: 647ea4aae4d52fe0e021bce4
paper_authors.avatarUrl: /avatars/9e68004d04403acff004113c856451a6.svg
paper_authors.fullname: Jiapeng Luo
paper_authors.type: user
paper_authors.user: woolpeeker
paper_authors._id: 662b161666c46de6b3519b1d
paper_authors.name: Zheng Ma
paper_authors._id: 662b161666c46de6b3519b1e
paper_authors.name: Ji Ma
paper_authors._id: 662b161666c46de6b3519b1f
paper_authors.name: Jiaqi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-09T20:06:10.802Z
paper_authors._id: 64b4eec4faa3181a5eab9c46
paper_authors.avatarUrl: /avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
paper_authors.fullname: Jiaqi Wang
paper_authors.type: user
paper_authors.user: myownskyW7
paper_authors._id: 662b161666c46de6b3519b20
paper_authors.name: Xiaoyi Dong
paper_authors._id: 662b161666c46de6b3519b21
paper_authors.name: Hang Yan
paper_authors._id: 662b161666c46de6b3519b22
paper_authors.name: Hewei Guo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-24T09:28:17.423Z
paper_authors._id: 662f58bc5cf9dca4a2b225ca
paper_authors.avatarUrl: /avatars/4b4f989621ddf2882b3dcdc43de4daf0.svg
paper_authors.fullname: Hewei Guo
paper_authors.type: user
paper_authors.user: heweiguo
paper_authors._id: 662b161666c46de6b3519b23
paper_authors.name: Conghui He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T07:57:11.292Z
paper_authors._id: 63f9fca8d4349b157a109eec
paper_authors.avatarUrl: /avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
paper_authors.fullname: Conghui He
paper_authors.type: user
paper_authors.user: conghui
paper_authors._id: 662b161666c46de6b3519b24
paper_authors.name: Zhenjiang Jin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T07:57:25.259Z
paper_authors._id: 63e9f81e74f940d171e4981a
paper_authors.avatarUrl: /avatars/c88a218efab9eb6d634373d1db4e1d63.svg
paper_authors.fullname: Jin Zhen Jiang
paper_authors.type: user
paper_authors.user: jinzhenj
paper_authors._id: 662b161666c46de6b3519b25
paper_authors.name: Chao Xu
paper_authors._id: 662b161666c46de6b3519b26
paper_authors.name: Bin Wang
paper_authors._id: 662b161666c46de6b3519b27
paper_authors.name: Xingjian Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T09:04:08.678Z
paper_authors._id: 6434ffcd7d013020da39632d
paper_authors.avatarUrl: /avatars/9faf652efc54b902cfabb280ddb7dc88.svg
paper_authors.fullname: WEIXINGJIANG
paper_authors.type: user
paper_authors.user: Wesley0618
paper_authors._id: 662b161666c46de6b3519b28
paper_authors.name: Wei Li
paper_authors._id: 662b161666c46de6b3519b29
paper_authors.name: Wenjian Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T09:04:31.693Z
paper_authors._id: 633567aecb526f8818c0ef6e
paper_authors.avatarUrl: /avatars/cc60d93e142f25d287ce59c58fd501fe.svg
paper_authors.fullname: zhangwenjian
paper_authors.type: user
paper_authors.user: zwj
paper_authors._id: 662b161666c46de6b3519b2a
paper_authors.name: Lewei Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T09:04:40.969Z
paper_authors._id: 65ead3ea908526a39082e641
paper_authors.avatarUrl: /avatars/dcf870695fd56b06ca03d82f831e9019.svg
paper_authors.fullname: Lewei Lu
paper_authors.type: user
paper_authors.user: luotto
paper_authors._id: 662b161666c46de6b3519b2b
paper_authors.name: Xizhou Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T09:04:48.870Z
paper_authors._id: 64ae2359179421d320b1694b
paper_authors.avatarUrl: /avatars/c387a75191005bcaa473091de5383a10.svg
paper_authors.fullname: Xizhou Zhu
paper_authors.type: user
paper_authors.user: Einsiedler
paper_authors._id: 662b161666c46de6b3519b2c
paper_authors.name: Tong Lu
paper_authors._id: 662b161666c46de6b3519b2d
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T09:05:11.304Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 662b161666c46de6b3519b2e
paper_authors.name: Yu Qiao
paper_title: How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal
    Models with Open-Source Suites
paper_summary: In this report, we introduce InternVL 1.5, an open-source multimodal large
  language model (MLLM) to bridge the capability gap between open-source and
  proprietary commercial models in multimodal understanding. We introduce three
  simple improvements: (1) Strong Vision Encoder: we explored a continuous
  learning strategy for the large-scale vision foundation model -- InternViT-6B,
  boosting its visual understanding capabilities, and making it can be
  transferred and reused in different LLMs. (2) Dynamic High-Resolution: we
  divide images into tiles ranging from 1 to 40 of 448times448 pixels
  according to the aspect ratio and resolution of the input images, which
  supports up to 4K resolution input. (3) High-Quality Bilingual Dataset: we
  carefully collected a high-quality bilingual dataset that covers common scenes,
  document images, and annotated them with English and Chinese question-answer
  pairs, significantly enhancing performance in OCR- and Chinese-related tasks.
  We evaluate InternVL 1.5 through a series of benchmarks and comparative
  studies. Compared to both open-source and proprietary models, InternVL 1.5
  shows competitive performance, achieving state-of-the-art results in 8 of 18
  benchmarks. Code has been released at https://github.com/OpenGVLab/InternVL.
paper_ai_keywords: InternVL 1.5 multimodal large language model (MLLM) continuous learning strategy InternViT-6B visual understanding capabilities dynamic high-resolution tiles 1 to 40 of 448$\times$448 pixels 4K resolution high-quality bilingual dataset common scenes document images English and Chinese question-answer pairs OCR- and Chinese-related tasks state-of-the-art results