InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding
Multimodal Large Language Models (MLLMs) have experienced significant
advancements recently. Nevertheless, challenges persist in the accurate
recognition and comprehension of intricate details within high-resolution
images. Despite being indispensable for the development of robust MLLMs, this
area remains underinvestigated. To tackle this challenge, our work introduces
InfiMM-HD, a novel architecture specifically designed for processing images of
different resolutions with low computational overhead. This innovation
facilitates the enlargement of MLLMs to higher-resolution capabilities.
InfiMM-HD incorporates a cross-attention module and visual windows to reduce
computation costs. By integrating this architectural design with a four-stage
training pipeline, our model attains improved visual perception efficiently and
cost-effectively. Empirical study underscores the robustness and effectiveness
of InfiMM-HD, opening new avenues for exploration in related areas. Codes and
models can be found at https://huggingface.co/Infi-MM/infimm-hd
65e6b05f54c1cdd61b2d4999
Haogeng Liu
admin_assigned
2024-03-05T20:52:29.336Z
64f58c8bab97f9a83eca4963
/avatars/b525fa19dcb6c051fe3ae260d85926e5.svg
haogengliu
user
lllliuhhhhggg
65e6b05f54c1cdd61b2d499a
Quanzeng You
admin_assigned
2024-03-05T20:52:40.708Z
642bb8c87f152f6e72b8d3cd
https://cdn-avatars.huggingface.co/v1/production/uploads/642bb8c87f152f6e72b8d3cd/MXxTssocydbabGZ6XFIic.png
Quanzeng You
user
Ye27
65e6b05f54c1cdd61b2d499b
Xiaotian Han
claimed_verified
2024-03-05T20:40:20.316Z
650dde4ce14eeb01d42b37a1
https://cdn-avatars.huggingface.co/v1/production/uploads/650dde4ce14eeb01d42b37a1/n5Yv24uofZ2XJjXdYCrKd.png
Xiaotian Han
user
xiaotianhan
65e6b05f54c1cdd61b2d499c
Yiqi Wang
admin_assigned
2024-03-05T20:53:03.621Z
6566d9088556065b5268c666
/avatars/85e275c3a1e70c21045f13b0cefc93fb.svg
Yiqi Wang
user
Yi-Qi638
65e6b05f54c1cdd61b2d499d
Bohan Zhai
admin_assigned
2024-03-05T20:53:10.947Z
642fb0309b2484d7d857dbae
/avatars/7b10993e665198e80bd6627bf584ed71.svg
Bohan Zhai
user
Borise
65e6b05f54c1cdd61b2d499e
Yongfei Liu
admin_assigned
2024-03-05T20:53:20.145Z
6282e4b6f13477a48424e550
/avatars/cadf3afe9e3843819c5077690c4b1d8b.svg
Liu
user
Yongfei
65e6b05f54c1cdd61b2d499f
Yunzhe Tao
65e6b05f54c1cdd61b2d49a0
Huaibo Huang
admin_assigned
2024-03-05T20:53:34.759Z
642259feeaad1bcb28b44f55
/avatars/685791fc7e66d685e9446228c9e278b1.svg
Huaibo Huang
user
Bob072
65e6b05f54c1cdd61b2d49a1
Ran He
65e6b05f54c1cdd61b2d49a2
Hongxia Yang
admin_assigned
2024-03-05T20:54:04.673Z
63a35c978c0c89dcae3a0653
/avatars/c0bb1d52e31f716da4cfd078b0200cf9.svg
Hongxia Yang
user
firewater26
InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding
Multimodal Large Language Models (MLLMs) have experienced significant
advancements recently. Nevertheless, challenges persist in the accurate
recognition and comprehension of intricate details within high-resolution
images. Despite being indispensable for the development of robust MLLMs, this
area remains underinvestigated. To tackle this challenge, our work introduces
InfiMM-HD, a novel architecture specifically designed for processing images of
different resolutions with low computational overhead. This innovation
facilitates the enlargement of MLLMs to higher-resolution capabilities.
InfiMM-HD incorporates a cross-attention module and visual windows to reduce
computation costs. By integrating this architectural design with a four-stage
training pipeline, our model attains improved visual perception efficiently and
cost-effectively. Empirical study underscores the robustness and effectiveness
of InfiMM-HD, opening new avenues for exploration in related areas. Codes and
models can be found at https://huggingface.co/Infi-MM/infimm-hd
InfiMM-HD, a novel architecture with cross-attention and visual windows, enhances multimodal large language models for high-resolution image processing with efficient training and low computational overhead.
multimodal large language models MLLMs InfiMM-HD cross-attention module visual windows four-stage training pipeline