Where do Large Vision-Language Models Look at when Answering Questions?
Large Vision-Language Models (LVLMs) have shown promising performance in
vision-language understanding and reasoning tasks. However, their visual
understanding behaviors remain underexplored. A fundamental question arises: to
what extent do LVLMs rely on visual input, and which image regions contribute
to their responses? It is non-trivial to interpret the free-form generation of
LVLMs due to their complicated visual architecture (e.g., multiple encoders and
multi-resolution) and variable-length outputs. In this paper, we extend
existing heatmap visualization methods (e.g., iGOS++) to support LVLMs for
open-ended visual question answering. We propose a method to select visually
relevant tokens that reflect the relevance between generated answers and input
image. Furthermore, we conduct a comprehensive analysis of state-of-the-art
LVLMs on benchmarks designed to require visual information to answer. Our
findings offer several insights into LVLM behavior, including the relationship
between focus region and answer correctness, differences in visual attention
across architectures, and the impact of LLM scale on visual understanding. The
code and data are available at
https://github.com/bytedance/LVLM_Interpretation.
67ddcdd46dc3daedb5e5fe4a
Xiaoying Xing
67ddcdd46dc3daedb5e5fe4b
Chia-Wen Kuo
67ddcdd46dc3daedb5e5fe4c
Li Fuxin
67ddcdd46dc3daedb5e5fe4d
Yulei Niu
67ddcdd46dc3daedb5e5fe4e
Fan Chen
claimed_verified
2025-05-06T08:36:06.081Z
65c2a2f5b911dda8cb3ff13b
/avatars/29ad2ed1433be78e1f1ceefbe7463a4e.svg
Fan Chen
user
fanchenBYT
67ddcdd46dc3daedb5e5fe4f
Ming Li
67ddcdd46dc3daedb5e5fe50
Ying Wu
67ddcdd46dc3daedb5e5fe51
Longyin Wen
67ddcdd46dc3daedb5e5fe52
Sijie Zhu
extracted_confirmed
2025-03-21T20:41:23.891Z
65cbdea6d6c974694f09249a
/avatars/a317a1f545117e0699e1c56258980fd8.svg
Sijie Zhu
user
Zilence006
Where do Large Vision-Language Models Look at when Answering Questions?
Large Vision-Language Models (LVLMs) have shown promising performance in
vision-language understanding and reasoning tasks. However, their visual
understanding behaviors remain underexplored. A fundamental question arises: to
what extent do LVLMs rely on visual input, and which image regions contribute
to their responses? It is non-trivial to interpret the free-form generation of
LVLMs due to their complicated visual architecture (e.g., multiple encoders and
multi-resolution) and variable-length outputs. In this paper, we extend
existing heatmap visualization methods (e.g., iGOS++) to support LVLMs for
open-ended visual question answering. We propose a method to select visually
relevant tokens that reflect the relevance between generated answers and input
image. Furthermore, we conduct a comprehensive analysis of state-of-the-art
LVLMs on benchmarks designed to require visual information to answer. Our
findings offer several insights into LVLM behavior, including the relationship
between focus region and answer correctness, differences in visual attention
across architectures, and the impact of LLM scale on visual understanding. The
code and data are available at
https://github.com/bytedance/LVLM_Interpretation.
A method to analyze and interpret visual understanding behaviors in large vision-language models through heatmap visualization and token selection is proposed, offering insights into model performance and architectural differences.
Large Vision-Language Models LVLMs vision-language understanding visual input image regions heatmap visualization visually relevant tokens open-ended visual question answering state-of-the-art LVLMs focus region answer correctness visual attention LLM scale