title: InternVL3: Exploring Advanced Training and Test-Time Recipes for
    Open-Source Multimodal Models
summary: We introduce InternVL3, a significant advancement in the InternVL series
  featuring a native multimodal pre-training paradigm. Rather than adapting a
  text-only large language model (LLM) into a multimodal large language model
  (MLLM) that supports visual inputs, InternVL3 jointly acquires multimodal and
  linguistic capabilities from both diverse multimodal data and pure-text corpora
  during a single pre-training stage. This unified training paradigm effectively
  addresses the complexities and alignment challenges commonly encountered in
  conventional post-hoc training pipelines for MLLMs. To further improve
  performance and scalability, InternVL3 incorporates variable visual position
  encoding (V2PE) to support extended multimodal contexts, employs advanced
  post-training techniques such as supervised fine-tuning (SFT) and mixed
  preference optimization (MPO), and adopts test-time scaling strategies
  alongside an optimized training infrastructure. Extensive empirical evaluations
  demonstrate that InternVL3 delivers superior performance across a wide range of
  multi-modal tasks. In particular, InternVL3-78B achieves a score of 72.2 on the
  MMMU benchmark, setting a new state-of-the-art among open-source MLLMs. Its
  capabilities remain highly competitive with leading proprietary models,
  including ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Pro, while also
  maintaining strong pure-language proficiency. In pursuit of open-science
  principles, we will publicly release both the training data and model weights
  to foster further research and development in next-generation MLLMs.
paper_authors._id: 67fdd08bda7816922cb67e54
paper_authors.name: Jinguo Zhu
paper_authors._id: 67fdd08bda7816922cb67e55
paper_authors.name: Weiyun Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:09:23.250Z
paper_authors._id: 619507e7b74b6c591f794340
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
paper_authors.fullname: Weiyun Wang
paper_authors.type: user
paper_authors.user: Weiyun1025
paper_authors._id: 67fdd08bda7816922cb67e56
paper_authors.name: Zhe Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-23T08:37:54.102Z
paper_authors._id: 64119264f0f81eb569e0d569
paper_authors.avatarUrl: /avatars/00dccedea51ac7d2f398ca9d65a8e78b.svg
paper_authors.fullname: Zhe Chen
paper_authors.type: user
paper_authors.user: czczup
paper_authors._id: 67fdd08bda7816922cb67e57
paper_authors.name: Zhaoyang Liu
paper_authors._id: 67fdd08bda7816922cb67e58
paper_authors.name: Shenglong Ye
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:09:57.293Z
paper_authors._id: 64804866c7f87934d082bb25
paper_authors.avatarUrl: /avatars/41761226c79ac16e48d4c4cb84362adb.svg
paper_authors.fullname: Yeshenglong
paper_authors.type: user
paper_authors.user: Yeshenglong
paper_authors._id: 67fdd08bda7816922cb67e59
paper_authors.name: Lixin Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:10:04.312Z
paper_authors._id: 6541efc9109d78427198ea40
paper_authors.avatarUrl: /avatars/c1252dd7da2e53b0b6757bf392139cdf.svg
paper_authors.fullname: Lixin Gu
paper_authors.type: user
paper_authors.user: gulixin0922
paper_authors._id: 67fdd08bda7816922cb67e5a
paper_authors.name: Yuchen Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:10:30.238Z
paper_authors._id: 6495a4d6d1cec0e2c2c96cdd
paper_authors.avatarUrl: /avatars/e8b42301514094c8af6fa1e5fcb53119.svg
paper_authors.fullname: Duan Yuchen
paper_authors.type: user
paper_authors.user: duanyuchen
paper_authors._id: 67fdd08bda7816922cb67e5b
paper_authors.name: Hao Tian
paper_authors._id: 67fdd08bda7816922cb67e5c
paper_authors.name: Weijie Su
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:10:39.196Z
paper_authors._id: 63e4562f9db5da2dc1f3b520
paper_authors.avatarUrl: /avatars/f4eecf1396b05e1c72436e7026d85cef.svg
paper_authors.fullname: Weijie Su
paper_authors.type: user
paper_authors.user: jackroos
paper_authors._id: 67fdd08bda7816922cb67e5d
paper_authors.name: Jie Shao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T09:23:40.264Z
paper_authors._id: 644a2bccd9a3ae834104b065
paper_authors.avatarUrl: /avatars/ee2caf787796cca438349d10089bdfd1.svg
paper_authors.fullname: Jie Shao
paper_authors.type: user
paper_authors.user: hehesang
paper_authors._id: 67fdd08bda7816922cb67e5e
paper_authors.name: Zhangwei Gao
paper_authors._id: 67fdd08bda7816922cb67e5f
paper_authors.name: Erfei Cui
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:10:59.950Z
paper_authors._id: 6579b818563044badca392fc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6579b818563044badca392fc/XTKQ9Lhceibp9dnQADPQF.jpeg
paper_authors.fullname: cuierfei
paper_authors.type: user
paper_authors.user: cuierfei
paper_authors._id: 67fdd08bda7816922cb67e60
paper_authors.name: Yue Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:11:07.485Z
paper_authors._id: 6571382c7644d1128561cebe
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/bnJ-T3k_w1g1Mr7b7LglK.jpeg
paper_authors.fullname: Cao Yue
paper_authors.type: user
paper_authors.user: yuecao0119
paper_authors._id: 67fdd08bda7816922cb67e61
paper_authors.name: Yangzhou Liu
paper_authors._id: 67fdd08bda7816922cb67e62
paper_authors.name: Weiye Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-24T09:11:23.399Z
paper_authors._id: 649e5ee29420f68cf1c1470e
paper_authors.avatarUrl: /avatars/7f6d1ec4fb3f85351e88044016d8ab42.svg
paper_authors.fullname: Xu Wayen
paper_authors.type: user
paper_authors.user: wilye
paper_authors._id: 67fdd08bda7816922cb67e63
paper_authors.name: Hao Li
paper_authors._id: 67fdd08bda7816922cb67e64
paper_authors.name: Jiahao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T09:23:43.045Z
paper_authors._id: 664b4a748dd1bfb5a3a970fe
paper_authors.avatarUrl: /avatars/37aa9332ab3e8fbb6ae30b875a7e0e5a.svg
paper_authors.fullname: Jiahao Wang
paper_authors.type: user
paper_authors.user: GenuineWWD
paper_authors._id: 67fdd08bda7816922cb67e65
paper_authors.name: Han Lv
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:11:58.666Z
paper_authors._id: 67c6fd7d85d2167189fce0e4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/qpvzOJWcUJvD03l5yTBcN.jpeg
paper_authors.fullname: Han Lv
paper_authors.type: user
paper_authors.user: Hanrandom
paper_authors._id: 67fdd08bda7816922cb67e66
paper_authors.name: Dengnian Chen
paper_authors._id: 67fdd08bda7816922cb67e67
paper_authors.name: Songze Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:12:32.493Z
paper_authors._id: 6751a10224431db3bed1f701
paper_authors.avatarUrl: /avatars/06dcc5ba6cec8aff217a8bbc7a7d3b73.svg
paper_authors.fullname: Songze Li
paper_authors.type: user
paper_authors.user: CatCatCat36
paper_authors._id: 67fdd08bda7816922cb67e68
paper_authors.name: Yinan He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:12:39.591Z
paper_authors._id: 65b9d9961fe588f824fde191
paper_authors.avatarUrl: /avatars/a9245958cc998a4b4b870bf2490fdaee.svg
paper_authors.fullname: Yinan He
paper_authors.type: user
paper_authors.user: yinanhe
paper_authors._id: 67fdd08bda7816922cb67e69
paper_authors.name: Tan Jiang
paper_authors._id: 67fdd08bda7816922cb67e6a
paper_authors.name: Jiapeng Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:12:51.845Z
paper_authors._id: 647ea4aae4d52fe0e021bce4
paper_authors.avatarUrl: /avatars/9e68004d04403acff004113c856451a6.svg
paper_authors.fullname: Jiapeng Luo
paper_authors.type: user
paper_authors.user: woolpeeker
paper_authors._id: 67fdd08bda7816922cb67e6b
paper_authors.name: Yi Wang
paper_authors._id: 67fdd08bda7816922cb67e6c
paper_authors.name: Conghui He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:13:07.750Z
paper_authors._id: 63f9fca8d4349b157a109eec
paper_authors.avatarUrl: /avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
paper_authors.fullname: Conghui He
paper_authors.type: user
paper_authors.user: conghui
paper_authors._id: 67fdd08bda7816922cb67e6d
paper_authors.name: Botian Shi
paper_authors._id: 67fdd08bda7816922cb67e6e
paper_authors.name: Xingcheng Zhang
paper_authors._id: 67fdd08bda7816922cb67e6f
paper_authors.name: Wenqi Shao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:15:27.767Z
paper_authors._id: 64b3fd42eec33e27dcc4c941
paper_authors.avatarUrl: /avatars/5aa1a99468fa61d4b8b0e80b592c4e55.svg
paper_authors.fullname: Wenqi Shao
paper_authors.type: user
paper_authors.user: wqshao126
paper_authors._id: 67fdd08bda7816922cb67e70
paper_authors.name: Junjun He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:15:19.980Z
paper_authors._id: 66b593026cef22e6ba6adb8a
paper_authors.avatarUrl: /avatars/8a94d55b85177e84d65dd0bd537e335f.svg
paper_authors.fullname: JunjunHe
paper_authors.type: user
paper_authors.user: JunjunHe
paper_authors._id: 67fdd08bda7816922cb67e71
paper_authors.name: Yingtong Xiong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:15:12.302Z
paper_authors._id: 654395472cfe8660a3a492bb
paper_authors.avatarUrl: /avatars/2010370112200e83ced979d3d4c87735.svg
paper_authors.fullname: xiong
paper_authors.type: user
paper_authors.user: xiongyingtong
paper_authors._id: 67fdd08bda7816922cb67e72
paper_authors.name: Wenwen Qu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:14:47.178Z
paper_authors._id: 6684eb7045d8ceb446ffe9ff
paper_authors.avatarUrl: /avatars/30586f289031696253842fcd4a8788b9.svg
paper_authors.fullname: wenwenQu
paper_authors.type: user
paper_authors.user: wenwenQu
paper_authors._id: 67fdd08bda7816922cb67e73
paper_authors.name: Peng Sun
paper_authors._id: 67fdd08bda7816922cb67e74
paper_authors.name: Penglong Jiao
paper_authors._id: 67fdd08bda7816922cb67e75
paper_authors.name: Lijun Wu
paper_authors._id: 67fdd08bda7816922cb67e76
paper_authors.name: Kaipeng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:13:25.647Z
paper_authors._id: 63527f4e7d071f23d085ad45
paper_authors.avatarUrl: /avatars/99a51adef5673b3ac1a8c02eb47759c4.svg
paper_authors.fullname: KAIPENG ZHANG
paper_authors.type: user
paper_authors.user: kpzhang
paper_authors._id: 67fdd08bda7816922cb67e77
paper_authors.name: Huipeng Deng
paper_authors._id: 67fdd08bda7816922cb67e78
paper_authors.name: Jiaye Ge
paper_authors._id: 67fdd08bda7816922cb67e79
paper_authors.name: Kai Chen
paper_authors._id: 67fdd08bda7816922cb67e7a
paper_authors.name: Limin Wang
paper_authors._id: 67fdd08bda7816922cb67e7b
paper_authors.name: Min Dou
paper_authors._id: 67fdd08bda7816922cb67e7c
paper_authors.name: Lewei Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:14:05.716Z
paper_authors._id: 65ead3ea908526a39082e641
paper_authors.avatarUrl: /avatars/dcf870695fd56b06ca03d82f831e9019.svg
paper_authors.fullname: Lewei Lu
paper_authors.type: user
paper_authors.user: luotto
paper_authors._id: 67fdd08bda7816922cb67e7d
paper_authors.name: Xizhou Zhu
paper_authors._id: 67fdd08bda7816922cb67e7e
paper_authors.name: Tong Lu
paper_authors._id: 67fdd08bda7816922cb67e7f
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:13:59.084Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 67fdd08bda7816922cb67e80
paper_authors.name: Yu Qiao
paper_authors._id: 67fdd08bda7816922cb67e81
paper_authors.name: Jifeng Dai
paper_authors._id: 67fdd08bda7816922cb67e82
paper_authors.name: Wenhai Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:13:51.523Z
paper_authors._id: 64d1c560c0c627dfa71bdbe0
paper_authors.avatarUrl: /avatars/f42794fe25bffcd870a1bcee69b95298.svg
paper_authors.fullname: wenhai.wang
paper_authors.type: user
paper_authors.user: wangwhcore
paper_title: InternVL3: Exploring Advanced Training and Test-Time Recipes for
    Open-Source Multimodal Models
paper_summary: We introduce InternVL3, a significant advancement in the InternVL series
  featuring a native multimodal pre-training paradigm. Rather than adapting a
  text-only large language model (LLM) into a multimodal large language model
  (MLLM) that supports visual inputs, InternVL3 jointly acquires multimodal and
  linguistic capabilities from both diverse multimodal data and pure-text corpora
  during a single pre-training stage. This unified training paradigm effectively
  addresses the complexities and alignment challenges commonly encountered in
  conventional post-hoc training pipelines for MLLMs. To further improve
  performance and scalability, InternVL3 incorporates variable visual position
  encoding (V2PE) to support extended multimodal contexts, employs advanced
  post-training techniques such as supervised fine-tuning (SFT) and mixed
  preference optimization (MPO), and adopts test-time scaling strategies
  alongside an optimized training infrastructure. Extensive empirical evaluations
  demonstrate that InternVL3 delivers superior performance across a wide range of
  multi-modal tasks. In particular, InternVL3-78B achieves a score of 72.2 on the
  MMMU benchmark, setting a new state-of-the-art among open-source MLLMs. Its
  capabilities remain highly competitive with leading proprietary models,
  including ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Pro, while also
  maintaining strong pure-language proficiency. In pursuit of open-science
  principles, we will publicly release both the training data and model weights
  to foster further research and development in next-generation MLLMs.
paper_ai_summary: InternVL3 is a multimodal pre-trained language model that jointly learns from both multimodal data and text, improving performance and scalability through advanced techniques and setting a new state-of-the-art in multimodal tasks.
paper_ai_keywords: multimodal pre-training large language model multimodal large language model variable visual position encoding supervised fine-tuning mixed preference optimization test-time scaling MMLM pure-language proficiency MMOU benchmark