VERIFY: A Benchmark of Visual Explanation and Reasoning for
  Investigating Multimodal Reasoning Fidelity
Visual reasoning is central to human cognition, enabling individuals to
interpret and abstractly understand their environment. Although recent
Multimodal Large Language Models (MLLMs) have demonstrated impressive
performance across language and vision-language tasks, existing benchmarks
primarily measure recognition-based skills and inadequately assess true visual
reasoning capabilities. To bridge this critical gap, we introduce VERIFY, a
benchmark explicitly designed to isolate and rigorously evaluate the visual
reasoning capabilities of state-of-the-art MLLMs. VERIFY compels models to
reason primarily from visual information, providing minimal textual context to
reduce reliance on domain-specific knowledge and linguistic biases. Each
problem is accompanied by a human-annotated reasoning path, making it the first
to provide in-depth evaluation of model decision-making processes.
Additionally, we propose novel metrics that assess visual reasoning fidelity
beyond mere accuracy, highlighting critical imbalances in current model
reasoning patterns. Our comprehensive benchmarking of leading MLLMs uncovers
significant limitations, underscoring the need for a balanced and holistic
approach to both perception and reasoning. For more teaser and testing, visit
our project page (https://verify-eqh.pages.dev/).
67d8889acf12ecfbbb2f109c
Jing Bi
claimed_verified
2025-03-18T08:08:56.144Z
675cae2c127c72c5682a4df6
https://cdn-avatars.huggingface.co/v1/production/uploads/675cae2c127c72c5682a4df6/-F7uVyRXiWlRHpJ8QxrSV.png
jing bi
user
jing-bi
67d8889acf12ecfbbb2f109d
Junjia Guo
67d8889acf12ecfbbb2f109e
Susan Liang
claimed_verified
2025-03-20T19:17:23.273Z
65763434a4ee9a4fe7cfb156
/avatars/2c4a23ff309f750dd9c0d67bd9fd7abc.svg
Susan Liang
user
susanliang
67d8889acf12ecfbbb2f109f
Guangyu Sun
extracted_confirmed
2025-03-17T20:42:24.004Z
65970f61719cf10fc5c28d85
/avatars/1cad14d28e9a53f5283a139c856690df.svg
Guangyu Sun
user
imguangyu
67d8889acf12ecfbbb2f10a0
Luchuan Song
67d8889acf12ecfbbb2f10a1
Yunlong Tang
claimed_verified
2025-03-20T19:17:25.910Z
6344c87f0f69ad8aa61dfcf6
https://cdn-avatars.huggingface.co/v1/production/uploads/6344c87f0f69ad8aa61dfcf6/m4QiQ_4c6GfOIZZFICy7T.jpeg
Yolo Y. Tang
user
yunlong10
67d8889acf12ecfbbb2f10a2
Jinxi He
67d8889acf12ecfbbb2f10a3
Jiarui Wu
claimed_verified
2025-03-31T08:15:49.072Z
667041706d34fe626259bae2
https://cdn-avatars.huggingface.co/v1/production/uploads/667041706d34fe626259bae2/2bO_Zo6NidzbU_KmKuclV.jpeg
Jiarui Wu
user
jwu114
67d8889acf12ecfbbb2f10a4
Ali Vosoughi
67d8889acf12ecfbbb2f10a5
Chen Chen
67d8889acf12ecfbbb2f10a6
Chenliang Xu
VERIFY: A Benchmark of Visual Explanation and Reasoning for
  Investigating Multimodal Reasoning Fidelity
Visual reasoning is central to human cognition, enabling individuals to
interpret and abstractly understand their environment. Although recent
Multimodal Large Language Models (MLLMs) have demonstrated impressive
performance across language and vision-language tasks, existing benchmarks
primarily measure recognition-based skills and inadequately assess true visual
reasoning capabilities. To bridge this critical gap, we introduce VERIFY, a
benchmark explicitly designed to isolate and rigorously evaluate the visual
reasoning capabilities of state-of-the-art MLLMs. VERIFY compels models to
reason primarily from visual information, providing minimal textual context to
reduce reliance on domain-specific knowledge and linguistic biases. Each
problem is accompanied by a human-annotated reasoning path, making it the first
to provide in-depth evaluation of model decision-making processes.
Additionally, we propose novel metrics that assess visual reasoning fidelity
beyond mere accuracy, highlighting critical imbalances in current model
reasoning patterns. Our comprehensive benchmarking of leading MLLMs uncovers
significant limitations, underscoring the need for a balanced and holistic
approach to both perception and reasoning. For more teaser and testing, visit
our project page (https://verify-eqh.pages.dev/).
VERIFY is a benchmark designed to evaluate the visual reasoning capabilities of MLLMs by minimizing textual context and assessing reasoning paths and fidelity.
visual reasoning Multimodal Large Language Models (MLLMs) benchmark recognition-based skills human-annotated reasoning path visual reasoning fidelity