title: Where do Large Vision-Language Models Look at when Answering Questions?
summary: Large Vision-Language Models (LVLMs) have shown promising performance in
  vision-language understanding and reasoning tasks. However, their visual
  understanding behaviors remain underexplored. A fundamental question arises: to
  what extent do LVLMs rely on visual input, and which image regions contribute
  to their responses? It is non-trivial to interpret the free-form generation of
  LVLMs due to their complicated visual architecture (e.g., multiple encoders and
  multi-resolution) and variable-length outputs. In this paper, we extend
  existing heatmap visualization methods (e.g., iGOS++) to support LVLMs for
  open-ended visual question answering. We propose a method to select visually
  relevant tokens that reflect the relevance between generated answers and input
  image. Furthermore, we conduct a comprehensive analysis of state-of-the-art
  LVLMs on benchmarks designed to require visual information to answer. Our
  findings offer several insights into LVLM behavior, including the relationship
  between focus region and answer correctness, differences in visual attention
  across architectures, and the impact of LLM scale on visual understanding. The
  code and data are available at
  https://github.com/bytedance/LVLM_Interpretation.
paper_authors._id: 67ddcdd46dc3daedb5e5fe4a
paper_authors.name: Xiaoying Xing
paper_authors._id: 67ddcdd46dc3daedb5e5fe4b
paper_authors.name: Chia-Wen Kuo
paper_authors._id: 67ddcdd46dc3daedb5e5fe4c
paper_authors.name: Li Fuxin
paper_authors._id: 67ddcdd46dc3daedb5e5fe4d
paper_authors.name: Yulei Niu
paper_authors._id: 67ddcdd46dc3daedb5e5fe4e
paper_authors.name: Fan Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:36:06.081Z
paper_authors._id: 65c2a2f5b911dda8cb3ff13b
paper_authors.avatarUrl: /avatars/29ad2ed1433be78e1f1ceefbe7463a4e.svg
paper_authors.fullname: Fan Chen
paper_authors.type: user
paper_authors.user: fanchenBYT
paper_authors._id: 67ddcdd46dc3daedb5e5fe4f
paper_authors.name: Ming Li
paper_authors._id: 67ddcdd46dc3daedb5e5fe50
paper_authors.name: Ying Wu
paper_authors._id: 67ddcdd46dc3daedb5e5fe51
paper_authors.name: Longyin Wen
paper_authors._id: 67ddcdd46dc3daedb5e5fe52
paper_authors.name: Sijie Zhu
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-21T20:41:23.891Z
paper_authors._id: 65cbdea6d6c974694f09249a
paper_authors.avatarUrl: /avatars/a317a1f545117e0699e1c56258980fd8.svg
paper_authors.fullname: Sijie Zhu
paper_authors.type: user
paper_authors.user: Zilence006
paper_title: Where do Large Vision-Language Models Look at when Answering Questions?
paper_summary: Large Vision-Language Models (LVLMs) have shown promising performance in
  vision-language understanding and reasoning tasks. However, their visual
  understanding behaviors remain underexplored. A fundamental question arises: to
  what extent do LVLMs rely on visual input, and which image regions contribute
  to their responses? It is non-trivial to interpret the free-form generation of
  LVLMs due to their complicated visual architecture (e.g., multiple encoders and
  multi-resolution) and variable-length outputs. In this paper, we extend
  existing heatmap visualization methods (e.g., iGOS++) to support LVLMs for
  open-ended visual question answering. We propose a method to select visually
  relevant tokens that reflect the relevance between generated answers and input
  image. Furthermore, we conduct a comprehensive analysis of state-of-the-art
  LVLMs on benchmarks designed to require visual information to answer. Our
  findings offer several insights into LVLM behavior, including the relationship
  between focus region and answer correctness, differences in visual attention
  across architectures, and the impact of LLM scale on visual understanding. The
  code and data are available at
  https://github.com/bytedance/LVLM_Interpretation.
paper_ai_summary: A method to analyze and interpret visual understanding behaviors in large vision-language models through heatmap visualization and token selection is proposed, offering insights into model performance and architectural differences.
paper_ai_keywords: Large Vision-Language Models LVLMs vision-language understanding visual input image regions heatmap visualization visually relevant tokens open-ended visual question answering state-of-the-art LVLMs focus region answer correctness visual attention LLM scale