InternLM-XComposer2: Mastering Free-form Text-Image Composition and
  Comprehension in Vision-Language Large Model
We introduce InternLM-XComposer2, a cutting-edge vision-language model
excelling in free-form text-image composition and comprehension. This model
goes beyond conventional vision-language understanding, adeptly crafting
interleaved text-image content from diverse inputs like outlines, detailed
textual specifications, and reference images, enabling highly customizable
content creation. InternLM-XComposer2 proposes a Partial LoRA (PLoRA) approach
that applies additional LoRA parameters exclusively to image tokens to preserve
the integrity of pre-trained language knowledge, striking a balance between
precise vision understanding and text composition with literary talent.
Experimental results demonstrate the superiority of InternLM-XComposer2 based
on InternLM2-7B in producing high-quality long-text multi-modal content and its
exceptional vision-language understanding performance across various
benchmarks, where it not only significantly outperforms existing multimodal
models but also matches or even surpasses GPT-4V and Gemini Pro in certain
assessments. This highlights its remarkable proficiency in the realm of
multimodal understanding. The InternLM-XComposer2 model series with 7B
parameters are publicly available at
https://github.com/InternLM/InternLM-XComposer.
65b86307da874c1bc094ac1b
Xiaoyi Dong
65b86307da874c1bc094ac1c
Pan Zhang
admin_assigned
2024-01-30T09:00:26.505Z
6434fd13ce04fdb57e9d1779
/avatars/6574180d5c88aeb93446a60feb9b2845.svg
Pan Zhang
user
Gatorburger
65b86307da874c1bc094ac1d
Yuhang Zang
admin_assigned
2024-01-30T09:00:34.290Z
63859cf3b2906edaf83af9f0
https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
Yuhang Zang
user
yuhangzang
65b86307da874c1bc094ac1e
Yuhang Cao
admin_assigned
2024-01-30T09:00:48.705Z
65000bef18830fabea469fdd
/avatars/b320c77dfad039d9f9c54127f610d44f.svg
Cao Yuhang
user
yhcao
65b86307da874c1bc094ac1f
Bin Wang
claimed_verified
2024-04-03T09:03:54.001Z
63ae9ff5557befe297a76f90
https://cdn-avatars.huggingface.co/v1/production/uploads/1672388558183-noauth.jpeg
Bin Wang
user
wanderkid
65b86307da874c1bc094ac20
Linke Ouyang
65b86307da874c1bc094ac21
Xilin Wei
admin_assigned
2024-01-30T09:03:46.599Z
62eb70462f0f5e54df42f778
/avatars/456049dba67638d3cdb330cdf383f272.svg
Xilin Wei
user
Wiselnn
65b86307da874c1bc094ac22
Songyang Zhang
admin_assigned
2024-01-30T09:04:24.118Z
630716d11801ecc7d2595021
/avatars/2d36a880ce4a3cf7efc5ff3987dbeaf3.svg
Songyang Zhang
user
zsytony
65b86307da874c1bc094ac23
Haodong Duan
admin_assigned
2024-01-30T09:05:06.108Z
63ee1379190ddd6214efd73a
https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
HAODONG DUAN
user
KennyUTC
65b86307da874c1bc094ac24
Maosong Cao
admin_assigned
2024-01-30T09:05:38.821Z
641585548b8245b157e15c5a
/avatars/c01a98d57733cfc29b549b974d4e2da4.svg
maosongcao
user
bittersweet
65b86307da874c1bc094ac25
Wenwei Zhang
admin_assigned
2024-01-30T09:05:45.967Z
64e8505321540e1da3226b54
/avatars/18958b8406d1ce492b54c1c839f18c54.svg
Wenwei Zhang
user
ZwwWayne
65b86307da874c1bc094ac26
Yining Li
admin_assigned
2024-01-30T09:06:10.410Z
649988726677f66c2b486392
/avatars/b649a77370660e129726e29504daba34.svg
Yining Li
user
ly015
65b86307da874c1bc094ac27
Hang Yan
65b86307da874c1bc094ac28
Yang Gao
65b86307da874c1bc094ac29
Xinyue Zhang
65b86307da874c1bc094ac2a
Wei Li
65b86307da874c1bc094ac2b
Jingwen Li
65b86307da874c1bc094ac2c
Kai Chen
65b86307da874c1bc094ac2d
Conghui He
admin_assigned
2024-01-30T09:10:43.768Z
63f9fca8d4349b157a109eec
/avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
Conghui He
user
conghui
65b86307da874c1bc094ac2e
Xingcheng Zhang
65b86307da874c1bc094ac2f
Yu Qiao
65b86307da874c1bc094ac30
Dahua Lin
admin_assigned
2024-01-30T09:11:32.155Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
65b86307da874c1bc094ac31
Jiaqi Wang
admin_assigned
2024-01-30T09:12:44.939Z
64b4eec4faa3181a5eab9c46
/avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
Jiaqi Wang
user
myownskyW7
InternLM-XComposer2: Mastering Free-form Text-Image Composition and
  Comprehension in Vision-Language Large Model
We introduce InternLM-XComposer2, a cutting-edge vision-language model
excelling in free-form text-image composition and comprehension. This model
goes beyond conventional vision-language understanding, adeptly crafting
interleaved text-image content from diverse inputs like outlines, detailed
textual specifications, and reference images, enabling highly customizable
content creation. InternLM-XComposer2 proposes a Partial LoRA (PLoRA) approach
that applies additional LoRA parameters exclusively to image tokens to preserve
the integrity of pre-trained language knowledge, striking a balance between
precise vision understanding and text composition with literary talent.
Experimental results demonstrate the superiority of InternLM-XComposer2 based
on InternLM2-7B in producing high-quality long-text multi-modal content and its
exceptional vision-language understanding performance across various
benchmarks, where it not only significantly outperforms existing multimodal
models but also matches or even surpasses GPT-4V and Gemini Pro in certain
assessments. This highlights its remarkable proficiency in the realm of
multimodal understanding. The InternLM-XComposer2 model series with 7B
parameters are publicly available at
https://github.com/InternLM/InternLM-XComposer.
InternLM-XComposer2, a vision-language model, excels in free-form text-image creation and understanding using Partial LoRA, surpassing existing models and matching GPT-4V and Gemini Pro in multimodal benchmarks.
vision-language model free-form text-image composition comprehension Partial LoRA (PLoRA) image tokens pre-trained language knowledge multimodal content vision-language understanding GPT-4V Gemini Pro multimodal understanding