title: InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding
summary: Multimodal Large Language Models (MLLMs) have experienced significant
  advancements recently. Nevertheless, challenges persist in the accurate
  recognition and comprehension of intricate details within high-resolution
  images. Despite being indispensable for the development of robust MLLMs, this
  area remains underinvestigated. To tackle this challenge, our work introduces
  InfiMM-HD, a novel architecture specifically designed for processing images of
  different resolutions with low computational overhead. This innovation
  facilitates the enlargement of MLLMs to higher-resolution capabilities.
  InfiMM-HD incorporates a cross-attention module and visual windows to reduce
  computation costs. By integrating this architectural design with a four-stage
  training pipeline, our model attains improved visual perception efficiently and
  cost-effectively. Empirical study underscores the robustness and effectiveness
  of InfiMM-HD, opening new avenues for exploration in related areas. Codes and
  models can be found at https://huggingface.co/Infi-MM/infimm-hd
paper_authors._id: 65e6b05f54c1cdd61b2d4999
paper_authors.name: Haogeng Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:52:29.336Z
paper_authors._id: 64f58c8bab97f9a83eca4963
paper_authors.avatarUrl: /avatars/b525fa19dcb6c051fe3ae260d85926e5.svg
paper_authors.fullname: haogengliu
paper_authors.type: user
paper_authors.user: lllliuhhhhggg
paper_authors._id: 65e6b05f54c1cdd61b2d499a
paper_authors.name: Quanzeng You
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:52:40.708Z
paper_authors._id: 642bb8c87f152f6e72b8d3cd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/642bb8c87f152f6e72b8d3cd/MXxTssocydbabGZ6XFIic.png
paper_authors.fullname: Quanzeng You
paper_authors.type: user
paper_authors.user: Ye27
paper_authors._id: 65e6b05f54c1cdd61b2d499b
paper_authors.name: Xiaotian Han
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-05T20:40:20.316Z
paper_authors._id: 650dde4ce14eeb01d42b37a1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650dde4ce14eeb01d42b37a1/n5Yv24uofZ2XJjXdYCrKd.png
paper_authors.fullname: Xiaotian Han
paper_authors.type: user
paper_authors.user: xiaotianhan
paper_authors._id: 65e6b05f54c1cdd61b2d499c
paper_authors.name: Yiqi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:53:03.621Z
paper_authors._id: 6566d9088556065b5268c666
paper_authors.avatarUrl: /avatars/85e275c3a1e70c21045f13b0cefc93fb.svg
paper_authors.fullname: Yiqi Wang
paper_authors.type: user
paper_authors.user: Yi-Qi638
paper_authors._id: 65e6b05f54c1cdd61b2d499d
paper_authors.name: Bohan Zhai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:53:10.947Z
paper_authors._id: 642fb0309b2484d7d857dbae
paper_authors.avatarUrl: /avatars/7b10993e665198e80bd6627bf584ed71.svg
paper_authors.fullname: Bohan Zhai
paper_authors.type: user
paper_authors.user: Borise
paper_authors._id: 65e6b05f54c1cdd61b2d499e
paper_authors.name: Yongfei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:53:20.145Z
paper_authors._id: 6282e4b6f13477a48424e550
paper_authors.avatarUrl: /avatars/cadf3afe9e3843819c5077690c4b1d8b.svg
paper_authors.fullname: Liu
paper_authors.type: user
paper_authors.user: Yongfei
paper_authors._id: 65e6b05f54c1cdd61b2d499f
paper_authors.name: Yunzhe Tao
paper_authors._id: 65e6b05f54c1cdd61b2d49a0
paper_authors.name: Huaibo Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:53:34.759Z
paper_authors._id: 642259feeaad1bcb28b44f55
paper_authors.avatarUrl: /avatars/685791fc7e66d685e9446228c9e278b1.svg
paper_authors.fullname: Huaibo Huang
paper_authors.type: user
paper_authors.user: Bob072
paper_authors._id: 65e6b05f54c1cdd61b2d49a1
paper_authors.name: Ran He
paper_authors._id: 65e6b05f54c1cdd61b2d49a2
paper_authors.name: Hongxia Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-05T20:54:04.673Z
paper_authors._id: 63a35c978c0c89dcae3a0653
paper_authors.avatarUrl: /avatars/c0bb1d52e31f716da4cfd078b0200cf9.svg
paper_authors.fullname: Hongxia Yang
paper_authors.type: user
paper_authors.user: firewater26
paper_title: InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding
paper_summary: Multimodal Large Language Models (MLLMs) have experienced significant
  advancements recently. Nevertheless, challenges persist in the accurate
  recognition and comprehension of intricate details within high-resolution
  images. Despite being indispensable for the development of robust MLLMs, this
  area remains underinvestigated. To tackle this challenge, our work introduces
  InfiMM-HD, a novel architecture specifically designed for processing images of
  different resolutions with low computational overhead. This innovation
  facilitates the enlargement of MLLMs to higher-resolution capabilities.
  InfiMM-HD incorporates a cross-attention module and visual windows to reduce
  computation costs. By integrating this architectural design with a four-stage
  training pipeline, our model attains improved visual perception efficiently and
  cost-effectively. Empirical study underscores the robustness and effectiveness
  of InfiMM-HD, opening new avenues for exploration in related areas. Codes and
  models can be found at https://huggingface.co/Infi-MM/infimm-hd
paper_ai_keywords: Multimodal Large Language Models (MLLMs) high-resolution images InfiMM-HD cross-attention module visual windows four-stage training pipeline visual perception