title: Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large
    Language Models
summary: Despite remarkable progress, existing multimodal large language models
  (MLLMs) are still inferior in granular visual recognition. Contrary to previous
  works, we study this problem from the perspective of image resolution, and
  reveal that a combination of low- and high-resolution visual features can
  effectively mitigate this shortcoming. Based on this observation, we propose a
  novel and efficient method for MLLMs, termed Mixture-of-Resolution Adaptation
  (MRA). In particular, MRA adopts two visual pathways for images with different
  resolutions, where high-resolution visual information is embedded into the
  low-resolution pathway via the novel mixture-of-resolution adapters
  (MR-Adapters). This design also greatly reduces the input sequence length of
  MLLMs. To validate MRA, we apply it to a recent MLLM called LLaVA, and term the
  new model LLaVA-HR. We conduct extensive experiments on 11 vision-language (VL)
  tasks, which show that LLaVA-HR outperforms existing MLLMs on 8 VL tasks, e.g.,
  +9.4% on TextVQA. More importantly, both training and inference of LLaVA-HR
  remain efficient with MRA, e.g., 20 training hours and 3times inference
  speed than LLaVA-1.5. Source codes are released at:
  https://github.com/luogen1996/LLaVA-HR.
paper_authors._id: 65e7debcb87e5297e4670304
paper_authors.name: Gen Luo
paper_authors._id: 65e7debcb87e5297e4670305
paper_authors.name: Yiyi Zhou
paper_authors._id: 65e7debcb87e5297e4670306
paper_authors.name: Yuxin Zhang
paper_authors._id: 65e7debcb87e5297e4670307
paper_authors.name: Xiawu Zheng
paper_authors._id: 65e7debcb87e5297e4670308
paper_authors.name: Xiaoshuai Sun
paper_authors._id: 65e7debcb87e5297e4670309
paper_authors.name: Rongrong Ji
paper_title: Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large
    Language Models
paper_summary: Despite remarkable progress, existing multimodal large language models
  (MLLMs) are still inferior in granular visual recognition. Contrary to previous
  works, we study this problem from the perspective of image resolution, and
  reveal that a combination of low- and high-resolution visual features can
  effectively mitigate this shortcoming. Based on this observation, we propose a
  novel and efficient method for MLLMs, termed Mixture-of-Resolution Adaptation
  (MRA). In particular, MRA adopts two visual pathways for images with different
  resolutions, where high-resolution visual information is embedded into the
  low-resolution pathway via the novel mixture-of-resolution adapters
  (MR-Adapters). This design also greatly reduces the input sequence length of
  MLLMs. To validate MRA, we apply it to a recent MLLM called LLaVA, and term the
  new model LLaVA-HR. We conduct extensive experiments on 11 vision-language (VL)
  tasks, which show that LLaVA-HR outperforms existing MLLMs on 8 VL tasks, e.g.,
  +9.4% on TextVQA. More importantly, both training and inference of LLaVA-HR
  remain efficient with MRA, e.g., 20 training hours and 3times inference
  speed than LLaVA-1.5. Source codes are released at:
  https://github.com/luogen1996/LLaVA-HR.
paper_ai_keywords: multimodal large language models (MLLMs) granular visual recognition image resolution high-resolution visual features low-resolution visual features Mixture-of-Resolution Adaptation (MRA) visual pathways high-resolution visual information low-resolution pathway mixture-of-resolution adapters (MR-Adapters) input sequence length LLaVA LLaVA-HR vision-language (VL) tasks TextVQA training efficiency inference speed