title: InternLM-XComposer2: Mastering Free-form Text-Image Composition and
    Comprehension in Vision-Language Large Model
summary: We introduce InternLM-XComposer2, a cutting-edge vision-language model
  excelling in free-form text-image composition and comprehension. This model
  goes beyond conventional vision-language understanding, adeptly crafting
  interleaved text-image content from diverse inputs like outlines, detailed
  textual specifications, and reference images, enabling highly customizable
  content creation. InternLM-XComposer2 proposes a Partial LoRA (PLoRA) approach
  that applies additional LoRA parameters exclusively to image tokens to preserve
  the integrity of pre-trained language knowledge, striking a balance between
  precise vision understanding and text composition with literary talent.
  Experimental results demonstrate the superiority of InternLM-XComposer2 based
  on InternLM2-7B in producing high-quality long-text multi-modal content and its
  exceptional vision-language understanding performance across various
  benchmarks, where it not only significantly outperforms existing multimodal
  models but also matches or even surpasses GPT-4V and Gemini Pro in certain
  assessments. This highlights its remarkable proficiency in the realm of
  multimodal understanding. The InternLM-XComposer2 model series with 7B
  parameters are publicly available at
  https://github.com/InternLM/InternLM-XComposer.
paper_authors._id: 65b86307da874c1bc094ac1b
paper_authors.name: Xiaoyi Dong
paper_authors._id: 65b86307da874c1bc094ac1c
paper_authors.name: Pan Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:00:26.505Z
paper_authors._id: 6434fd13ce04fdb57e9d1779
paper_authors.avatarUrl: /avatars/6574180d5c88aeb93446a60feb9b2845.svg
paper_authors.fullname: Pan Zhang
paper_authors.type: user
paper_authors.user: Gatorburger
paper_authors._id: 65b86307da874c1bc094ac1d
paper_authors.name: Yuhang Zang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:00:34.290Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 65b86307da874c1bc094ac1e
paper_authors.name: Yuhang Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:00:48.705Z
paper_authors._id: 65000bef18830fabea469fdd
paper_authors.avatarUrl: /avatars/b320c77dfad039d9f9c54127f610d44f.svg
paper_authors.fullname: Cao Yuhang
paper_authors.type: user
paper_authors.user: yhcao
paper_authors._id: 65b86307da874c1bc094ac1f
paper_authors.name: Bin Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-03T09:03:54.001Z
paper_authors._id: 63ae9ff5557befe297a76f90
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1672388558183-noauth.jpeg
paper_authors.fullname: Bin Wang
paper_authors.type: user
paper_authors.user: wanderkid
paper_authors._id: 65b86307da874c1bc094ac20
paper_authors.name: Linke Ouyang
paper_authors._id: 65b86307da874c1bc094ac21
paper_authors.name: Xilin Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:03:46.599Z
paper_authors._id: 62eb70462f0f5e54df42f778
paper_authors.avatarUrl: /avatars/456049dba67638d3cdb330cdf383f272.svg
paper_authors.fullname: Xilin Wei
paper_authors.type: user
paper_authors.user: Wiselnn
paper_authors._id: 65b86307da874c1bc094ac22
paper_authors.name: Songyang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:04:24.118Z
paper_authors._id: 630716d11801ecc7d2595021
paper_authors.avatarUrl: /avatars/2d36a880ce4a3cf7efc5ff3987dbeaf3.svg
paper_authors.fullname: Songyang Zhang
paper_authors.type: user
paper_authors.user: zsytony
paper_authors._id: 65b86307da874c1bc094ac23
paper_authors.name: Haodong Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:05:06.108Z
paper_authors._id: 63ee1379190ddd6214efd73a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
paper_authors.fullname: HAODONG DUAN
paper_authors.type: user
paper_authors.user: KennyUTC
paper_authors._id: 65b86307da874c1bc094ac24
paper_authors.name: Maosong Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:05:38.821Z
paper_authors._id: 641585548b8245b157e15c5a
paper_authors.avatarUrl: /avatars/c01a98d57733cfc29b549b974d4e2da4.svg
paper_authors.fullname: maosongcao
paper_authors.type: user
paper_authors.user: bittersweet
paper_authors._id: 65b86307da874c1bc094ac25
paper_authors.name: Wenwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:05:45.967Z
paper_authors._id: 64e8505321540e1da3226b54
paper_authors.avatarUrl: /avatars/18958b8406d1ce492b54c1c839f18c54.svg
paper_authors.fullname: Wenwei Zhang
paper_authors.type: user
paper_authors.user: ZwwWayne
paper_authors._id: 65b86307da874c1bc094ac26
paper_authors.name: Yining Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:06:10.410Z
paper_authors._id: 649988726677f66c2b486392
paper_authors.avatarUrl: /avatars/b649a77370660e129726e29504daba34.svg
paper_authors.fullname: Yining Li
paper_authors.type: user
paper_authors.user: ly015
paper_authors._id: 65b86307da874c1bc094ac27
paper_authors.name: Hang Yan
paper_authors._id: 65b86307da874c1bc094ac28
paper_authors.name: Yang Gao
paper_authors._id: 65b86307da874c1bc094ac29
paper_authors.name: Xinyue Zhang
paper_authors._id: 65b86307da874c1bc094ac2a
paper_authors.name: Wei Li
paper_authors._id: 65b86307da874c1bc094ac2b
paper_authors.name: Jingwen Li
paper_authors._id: 65b86307da874c1bc094ac2c
paper_authors.name: Kai Chen
paper_authors._id: 65b86307da874c1bc094ac2d
paper_authors.name: Conghui He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:10:43.768Z
paper_authors._id: 63f9fca8d4349b157a109eec
paper_authors.avatarUrl: /avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
paper_authors.fullname: Conghui He
paper_authors.type: user
paper_authors.user: conghui
paper_authors._id: 65b86307da874c1bc094ac2e
paper_authors.name: Xingcheng Zhang
paper_authors._id: 65b86307da874c1bc094ac2f
paper_authors.name: Yu Qiao
paper_authors._id: 65b86307da874c1bc094ac30
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:11:32.155Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 65b86307da874c1bc094ac31
paper_authors.name: Jiaqi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-30T09:12:44.939Z
paper_authors._id: 64b4eec4faa3181a5eab9c46
paper_authors.avatarUrl: /avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
paper_authors.fullname: Jiaqi Wang
paper_authors.type: user
paper_authors.user: myownskyW7
paper_title: InternLM-XComposer2: Mastering Free-form Text-Image Composition and
    Comprehension in Vision-Language Large Model
paper_summary: We introduce InternLM-XComposer2, a cutting-edge vision-language model
  excelling in free-form text-image composition and comprehension. This model
  goes beyond conventional vision-language understanding, adeptly crafting
  interleaved text-image content from diverse inputs like outlines, detailed
  textual specifications, and reference images, enabling highly customizable
  content creation. InternLM-XComposer2 proposes a Partial LoRA (PLoRA) approach
  that applies additional LoRA parameters exclusively to image tokens to preserve
  the integrity of pre-trained language knowledge, striking a balance between
  precise vision understanding and text composition with literary talent.
  Experimental results demonstrate the superiority of InternLM-XComposer2 based
  on InternLM2-7B in producing high-quality long-text multi-modal content and its
  exceptional vision-language understanding performance across various
  benchmarks, where it not only significantly outperforms existing multimodal
  models but also matches or even surpasses GPT-4V and Gemini Pro in certain
  assessments. This highlights its remarkable proficiency in the realm of
  multimodal understanding. The InternLM-XComposer2 model series with 7B
  parameters are publicly available at
  https://github.com/InternLM/InternLM-XComposer.
paper_ai_summary: InternLM-XComposer2, a vision-language model, excels in free-form text-image creation and understanding using Partial LoRA, surpassing existing models and matching GPT-4V and Gemini Pro in multimodal benchmarks.
paper_ai_keywords: vision-language model free-form text-image composition comprehension Partial LoRA (PLoRA) image tokens pre-trained language knowledge multimodal content vision-language understanding GPT-4V Gemini Pro multimodal understanding