How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal
  Models with Open-Source Suites
In this report, we introduce InternVL 1.5, an open-source multimodal large
language model (MLLM) to bridge the capability gap between open-source and
proprietary commercial models in multimodal understanding. We introduce three
simple improvements: (1) Strong Vision Encoder: we explored a continuous
learning strategy for the large-scale vision foundation model -- InternViT-6B,
boosting its visual understanding capabilities, and making it can be
transferred and reused in different LLMs. (2) Dynamic High-Resolution: we
divide images into tiles ranging from 1 to 40 of 448times448 pixels
according to the aspect ratio and resolution of the input images, which
supports up to 4K resolution input. (3) High-Quality Bilingual Dataset: we
carefully collected a high-quality bilingual dataset that covers common scenes,
document images, and annotated them with English and Chinese question-answer
pairs, significantly enhancing performance in OCR- and Chinese-related tasks.
We evaluate InternVL 1.5 through a series of benchmarks and comparative
studies. Compared to both open-source and proprietary models, InternVL 1.5
shows competitive performance, achieving state-of-the-art results in 8 of 18
benchmarks. Code has been released at https://github.com/OpenGVLab/InternVL.
662b161666c46de6b3519b14
Zhe Chen
claimed_verified
2024-12-09T14:49:44.356Z
64119264f0f81eb569e0d569
/avatars/00dccedea51ac7d2f398ca9d65a8e78b.svg
Zhe Chen
user
czczup
662b161666c46de6b3519b15
Weiyun Wang
admin_assigned
2024-04-26T07:49:56.737Z
619507e7b74b6c591f794340
https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
Weiyun Wang
user
Weiyun1025
662b161666c46de6b3519b16
Hao Tian
662b161666c46de6b3519b17
Shenglong Ye
admin_assigned
2024-04-26T07:52:57.771Z
64804866c7f87934d082bb25
/avatars/41761226c79ac16e48d4c4cb84362adb.svg
Yeshenglong
user
Yeshenglong
662b161666c46de6b3519b18
Zhangwei Gao
662b161666c46de6b3519b19
Erfei Cui
admin_assigned
2024-04-26T07:53:24.828Z
6579b818563044badca392fc
https://cdn-avatars.huggingface.co/v1/production/uploads/6579b818563044badca392fc/XTKQ9Lhceibp9dnQADPQF.jpeg
cuierfei
user
cuierfei
662b161666c46de6b3519b1a
Wenwen Tong
admin_assigned
2024-04-26T07:53:31.787Z
647d4f1236e109abce409c3b
/avatars/d166f5f8be666e96b522a0a0effd21c4.svg
Wenwen Tong
user
tongww
662b161666c46de6b3519b1b
Kongzhi Hu
662b161666c46de6b3519b1c
Jiapeng Luo
admin_assigned
2024-04-26T07:53:46.130Z
647ea4aae4d52fe0e021bce4
/avatars/9e68004d04403acff004113c856451a6.svg
Jiapeng Luo
user
woolpeeker
662b161666c46de6b3519b1d
Zheng Ma
662b161666c46de6b3519b1e
Ji Ma
662b161666c46de6b3519b1f
Jiaqi Wang
claimed_verified
2024-06-09T20:06:10.802Z
64b4eec4faa3181a5eab9c46
/avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
Jiaqi Wang
user
myownskyW7
662b161666c46de6b3519b20
Xiaoyi Dong
662b161666c46de6b3519b21
Hang Yan
662b161666c46de6b3519b22
Hewei Guo
claimed_verified
2025-02-24T09:28:17.423Z
662f58bc5cf9dca4a2b225ca
/avatars/4b4f989621ddf2882b3dcdc43de4daf0.svg
Hewei Guo
user
heweiguo
662b161666c46de6b3519b23
Conghui He
admin_assigned
2024-04-26T07:57:11.292Z
63f9fca8d4349b157a109eec
/avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
Conghui He
user
conghui
662b161666c46de6b3519b24
Zhenjiang Jin
admin_assigned
2024-04-26T07:57:25.259Z
63e9f81e74f940d171e4981a
/avatars/c88a218efab9eb6d634373d1db4e1d63.svg
Jin Zhen Jiang
user
jinzhenj
662b161666c46de6b3519b25
Chao Xu
662b161666c46de6b3519b26
Bin Wang
662b161666c46de6b3519b27
Xingjian Wei
admin_assigned
2024-04-26T09:04:08.678Z
6434ffcd7d013020da39632d
/avatars/9faf652efc54b902cfabb280ddb7dc88.svg
WEIXINGJIANG
user
Wesley0618
662b161666c46de6b3519b28
Wei Li
662b161666c46de6b3519b29
Wenjian Zhang
admin_assigned
2024-04-26T09:04:31.693Z
633567aecb526f8818c0ef6e
/avatars/cc60d93e142f25d287ce59c58fd501fe.svg
zhangwenjian
user
zwj
662b161666c46de6b3519b2a
Lewei Lu
admin_assigned
2024-04-26T09:04:40.969Z
65ead3ea908526a39082e641
/avatars/dcf870695fd56b06ca03d82f831e9019.svg
Lewei Lu
user
luotto
662b161666c46de6b3519b2b
Xizhou Zhu
admin_assigned
2024-04-26T09:04:48.870Z
64ae2359179421d320b1694b
/avatars/c387a75191005bcaa473091de5383a10.svg
Xizhou Zhu
user
Einsiedler
662b161666c46de6b3519b2c
Tong Lu
662b161666c46de6b3519b2d
Dahua Lin
admin_assigned
2024-04-26T09:05:11.304Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
662b161666c46de6b3519b2e
Yu Qiao
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal
  Models with Open-Source Suites
In this report, we introduce InternVL 1.5, an open-source multimodal large
language model (MLLM) to bridge the capability gap between open-source and
proprietary commercial models in multimodal understanding. We introduce three
simple improvements: (1) Strong Vision Encoder: we explored a continuous
learning strategy for the large-scale vision foundation model -- InternViT-6B,
boosting its visual understanding capabilities, and making it can be
transferred and reused in different LLMs. (2) Dynamic High-Resolution: we
divide images into tiles ranging from 1 to 40 of 448times448 pixels
according to the aspect ratio and resolution of the input images, which
supports up to 4K resolution input. (3) High-Quality Bilingual Dataset: we
carefully collected a high-quality bilingual dataset that covers common scenes,
document images, and annotated them with English and Chinese question-answer
pairs, significantly enhancing performance in OCR- and Chinese-related tasks.
We evaluate InternVL 1.5 through a series of benchmarks and comparative
studies. Compared to both open-source and proprietary models, InternVL 1.5
shows competitive performance, achieving state-of-the-art results in 8 of 18
benchmarks. Code has been released at https://github.com/OpenGVLab/InternVL.
InternVL 1.5, an open-source multimodal language model, improves visual understanding and high-resolution image processing through continuous learning, dynamic tile division, and a high-quality bilingual dataset, achieving state-of-the-art results in several benchmarks.
multimodal large language model MLLM continuous learning InternViT-2B dynamic high-resolution image tiles high-quality bilingual dataset OCR Chinese-related tasks