title: InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model
    Handling Resolutions from 336 Pixels to 4K HD
summary: The Large Vision-Language Model (LVLM) field has seen significant
  advancements, yet its progression has been hindered by challenges in
  comprehending fine-grained visual content due to limited resolution. Recent
  efforts have aimed to enhance the high-resolution understanding capabilities of
  LVLMs, yet they remain capped at approximately 1500 x 1500 pixels and
  constrained to a relatively narrow resolution range. This paper represents
  InternLM-XComposer2-4KHD, a groundbreaking exploration into elevating LVLM
  resolution capabilities up to 4K HD (3840 x 1600) and beyond. Concurrently,
  considering the ultra-high resolution may not be necessary in all scenarios, it
  supports a wide range of diverse resolutions from 336 pixels to 4K standard,
  significantly broadening its scope of applicability. Specifically, this
  research advances the patch division paradigm by introducing a novel extension:
  dynamic resolution with automatic patch configuration. It maintains the
  training image aspect ratios while automatically varying patch counts and
  configuring layouts based on a pre-trained Vision Transformer (ViT) (336 x
  336), leading to dynamic training resolution from 336 pixels to 4K standard.
  Our research demonstrates that scaling training resolution up to 4K HD leads to
  consistent performance enhancements without hitting the ceiling of potential
  improvements. InternLM-XComposer2-4KHD shows superb capability that matches or
  even surpasses GPT-4V and Gemini Pro in 10 of the 16 benchmarks. The
  InternLM-XComposer2-4KHD model series with 7B parameters are publicly available
  at https://github.com/InternLM/InternLM-XComposer.
paper_authors._id: 6615f0031ddd756333b2f134
paper_authors.name: Xiaoyi Dong
paper_authors._id: 6615f0031ddd756333b2f135
paper_authors.name: Pan Zhang
paper_authors._id: 6615f0031ddd756333b2f136
paper_authors.name: Yuhang Zang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-22T14:09:41.451Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/iUQm5FAomzqYi6fkqIn9F.jpeg
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 6615f0031ddd756333b2f137
paper_authors.name: Yuhang Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-22T14:09:51.134Z
paper_authors._id: 65000bef18830fabea469fdd
paper_authors.avatarUrl: /avatars/b320c77dfad039d9f9c54127f610d44f.svg
paper_authors.fullname: Cao Yuhang
paper_authors.type: user
paper_authors.user: yhcao
paper_authors._id: 6615f0031ddd756333b2f138
paper_authors.name: Bin Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-22T14:09:57.885Z
paper_authors._id: 5e49e8cf37cb5b49818287ae
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5e49e8cf37cb5b49818287ae/IV9b5Z70NhgmBNfAlc_co.jpeg
paper_authors.fullname: Bin Wang
paper_authors.type: user
paper_authors.user: binwang
paper_authors._id: 6615f0031ddd756333b2f139
paper_authors.name: Linke Ouyang
paper_authors._id: 6615f0031ddd756333b2f13a
paper_authors.name: Songyang Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-05T08:00:27.592Z
paper_authors._id: 630716d11801ecc7d2595021
paper_authors.avatarUrl: /avatars/2d36a880ce4a3cf7efc5ff3987dbeaf3.svg
paper_authors.fullname: Songyang Zhang
paper_authors.type: user
paper_authors.user: zsytony
paper_authors._id: 6615f0031ddd756333b2f13b
paper_authors.name: Haodong Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-22T14:10:39.088Z
paper_authors._id: 63ee1379190ddd6214efd73a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
paper_authors.fullname: HAODONG DUAN
paper_authors.type: user
paper_authors.user: KennyUTC
paper_authors._id: 6615f0031ddd756333b2f13c
paper_authors.name: Wenwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-22T14:10:46.298Z
paper_authors._id: 64e8505321540e1da3226b54
paper_authors.avatarUrl: /avatars/18958b8406d1ce492b54c1c839f18c54.svg
paper_authors.fullname: Wenwei Zhang
paper_authors.type: user
paper_authors.user: ZwwWayne
paper_authors._id: 6615f0031ddd756333b2f13d
paper_authors.name: Yining Li
paper_authors._id: 6615f0031ddd756333b2f13e
paper_authors.name: Hang Yan
paper_authors._id: 6615f0031ddd756333b2f13f
paper_authors.name: Yang Gao
paper_authors._id: 6615f0031ddd756333b2f140
paper_authors.name: Zhe Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-09T14:49:47.535Z
paper_authors._id: 64119264f0f81eb569e0d569
paper_authors.avatarUrl: /avatars/00dccedea51ac7d2f398ca9d65a8e78b.svg
paper_authors.fullname: Zhe Chen
paper_authors.type: user
paper_authors.user: czczup
paper_authors._id: 6615f0031ddd756333b2f141
paper_authors.name: Xinyue Zhang
paper_authors._id: 6615f0031ddd756333b2f142
paper_authors.name: Wei Li
paper_authors._id: 6615f0031ddd756333b2f143
paper_authors.name: Jingwen Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-22T14:11:56.832Z
paper_authors._id: 64da1d1e7f8116a6ab1f5181
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/blOlCW3k-GR4zV9zCA94s.jpeg
paper_authors.fullname: Jingwen Li
paper_authors.type: user
paper_authors.user: mersso
paper_authors._id: 6615f0031ddd756333b2f144
paper_authors.name: Wenhai Wang
paper_authors._id: 6615f0031ddd756333b2f145
paper_authors.name: Kai Chen
paper_authors._id: 6615f0031ddd756333b2f146
paper_authors.name: Conghui He
paper_authors._id: 6615f0031ddd756333b2f147
paper_authors.name: Xingcheng Zhang
paper_authors._id: 6615f0031ddd756333b2f148
paper_authors.name: Jifeng Dai
paper_authors._id: 6615f0031ddd756333b2f149
paper_authors.name: Yu Qiao
paper_authors._id: 6615f0031ddd756333b2f14a
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-22T14:12:13.665Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 6615f0031ddd756333b2f14b
paper_authors.name: Jiaqi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-09T20:06:20.449Z
paper_authors._id: 64b4eec4faa3181a5eab9c46
paper_authors.avatarUrl: /avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
paper_authors.fullname: Jiaqi Wang
paper_authors.type: user
paper_authors.user: myownskyW7
paper_title: InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model
    Handling Resolutions from 336 Pixels to 4K HD
paper_summary: The Large Vision-Language Model (LVLM) field has seen significant
  advancements, yet its progression has been hindered by challenges in
  comprehending fine-grained visual content due to limited resolution. Recent
  efforts have aimed to enhance the high-resolution understanding capabilities of
  LVLMs, yet they remain capped at approximately 1500 x 1500 pixels and
  constrained to a relatively narrow resolution range. This paper represents
  InternLM-XComposer2-4KHD, a groundbreaking exploration into elevating LVLM
  resolution capabilities up to 4K HD (3840 x 1600) and beyond. Concurrently,
  considering the ultra-high resolution may not be necessary in all scenarios, it
  supports a wide range of diverse resolutions from 336 pixels to 4K standard,
  significantly broadening its scope of applicability. Specifically, this
  research advances the patch division paradigm by introducing a novel extension:
  dynamic resolution with automatic patch configuration. It maintains the
  training image aspect ratios while automatically varying patch counts and
  configuring layouts based on a pre-trained Vision Transformer (ViT) (336 x
  336), leading to dynamic training resolution from 336 pixels to 4K standard.
  Our research demonstrates that scaling training resolution up to 4K HD leads to
  consistent performance enhancements without hitting the ceiling of potential
  improvements. InternLM-XComposer2-4KHD shows superb capability that matches or
  even surpasses GPT-4V and Gemini Pro in 10 of the 16 benchmarks. The
  InternLM-XComposer2-4KHD model series with 7B parameters are publicly available
  at https://github.com/InternLM/InternLM-XComposer.
paper_ai_keywords: Large Vision-Language Model (LVLM) Fine-grained visual content High-resolution understanding capabilities Vision Transformer (ViT) Patch division paradigm Dynamic resolution Automatic patch configuration Training image aspect ratios Scaling training resolution Performance enhancements GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V GPT-4V G