title: VERIFY: A Benchmark of Visual Explanation and Reasoning for
    Investigating Multimodal Reasoning Fidelity
summary: Visual reasoning is central to human cognition, enabling individuals to
  interpret and abstractly understand their environment. Although recent
  Multimodal Large Language Models (MLLMs) have demonstrated impressive
  performance across language and vision-language tasks, existing benchmarks
  primarily measure recognition-based skills and inadequately assess true visual
  reasoning capabilities. To bridge this critical gap, we introduce VERIFY, a
  benchmark explicitly designed to isolate and rigorously evaluate the visual
  reasoning capabilities of state-of-the-art MLLMs. VERIFY compels models to
  reason primarily from visual information, providing minimal textual context to
  reduce reliance on domain-specific knowledge and linguistic biases. Each
  problem is accompanied by a human-annotated reasoning path, making it the first
  to provide in-depth evaluation of model decision-making processes.
  Additionally, we propose novel metrics that assess visual reasoning fidelity
  beyond mere accuracy, highlighting critical imbalances in current model
  reasoning patterns. Our comprehensive benchmarking of leading MLLMs uncovers
  significant limitations, underscoring the need for a balanced and holistic
  approach to both perception and reasoning. For more teaser and testing, visit
  our project page (https://verify-eqh.pages.dev/).
paper_authors._id: 67d8889acf12ecfbbb2f109c
paper_authors.name: Jing Bi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-18T08:08:56.144Z
paper_authors._id: 675cae2c127c72c5682a4df6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/675cae2c127c72c5682a4df6/Fpkm1ilmoIpjPR1j8ipSD.png
paper_authors.fullname: jing bi
paper_authors.type: user
paper_authors.user: jing-bi
paper_authors._id: 67d8889acf12ecfbbb2f109d
paper_authors.name: Junjia Guo
paper_authors._id: 67d8889acf12ecfbbb2f109e
paper_authors.name: Susan Liang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T19:17:23.273Z
paper_authors._id: 65763434a4ee9a4fe7cfb156
paper_authors.avatarUrl: /avatars/2c4a23ff309f750dd9c0d67bd9fd7abc.svg
paper_authors.fullname: Susan Liang
paper_authors.type: user
paper_authors.user: susanliang
paper_authors._id: 67d8889acf12ecfbbb2f109f
paper_authors.name: Guangyu Sun
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-17T20:42:24.004Z
paper_authors._id: 65970f61719cf10fc5c28d85
paper_authors.avatarUrl: /avatars/1cad14d28e9a53f5283a139c856690df.svg
paper_authors.fullname: Guangyu Sun
paper_authors.type: user
paper_authors.user: imguangyu
paper_authors._id: 67d8889acf12ecfbbb2f10a0
paper_authors.name: Luchuan Song
paper_authors._id: 67d8889acf12ecfbbb2f10a1
paper_authors.name: Yunlong Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T19:17:25.910Z
paper_authors._id: 6344c87f0f69ad8aa61dfcf6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6344c87f0f69ad8aa61dfcf6/m4QiQ_4c6GfOIZZFICy7T.jpeg
paper_authors.fullname: Yolo Y. Tang
paper_authors.type: user
paper_authors.user: yunlong10
paper_authors._id: 67d8889acf12ecfbbb2f10a2
paper_authors.name: Jinxi He
paper_authors._id: 67d8889acf12ecfbbb2f10a3
paper_authors.name: Jiarui Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-31T08:15:49.072Z
paper_authors._id: 667041706d34fe626259bae2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/667041706d34fe626259bae2/2bO_Zo6NidzbU_KmKuclV.jpeg
paper_authors.fullname: Jiarui Wu
paper_authors.type: user
paper_authors.user: jwu114
paper_authors._id: 67d8889acf12ecfbbb2f10a4
paper_authors.name: Ali Vosoughi
paper_authors._id: 67d8889acf12ecfbbb2f10a5
paper_authors.name: Chen Chen
paper_authors._id: 67d8889acf12ecfbbb2f10a6
paper_authors.name: Chenliang Xu
paper_title: VERIFY: A Benchmark of Visual Explanation and Reasoning for
    Investigating Multimodal Reasoning Fidelity
paper_summary: Visual reasoning is central to human cognition, enabling individuals to
  interpret and abstractly understand their environment. Although recent
  Multimodal Large Language Models (MLLMs) have demonstrated impressive
  performance across language and vision-language tasks, existing benchmarks
  primarily measure recognition-based skills and inadequately assess true visual
  reasoning capabilities. To bridge this critical gap, we introduce VERIFY, a
  benchmark explicitly designed to isolate and rigorously evaluate the visual
  reasoning capabilities of state-of-the-art MLLMs. VERIFY compels models to
  reason primarily from visual information, providing minimal textual context to
  reduce reliance on domain-specific knowledge and linguistic biases. Each
  problem is accompanied by a human-annotated reasoning path, making it the first
  to provide in-depth evaluation of model decision-making processes.
  Additionally, we propose novel metrics that assess visual reasoning fidelity
  beyond mere accuracy, highlighting critical imbalances in current model
  reasoning patterns. Our comprehensive benchmarking of leading MLLMs uncovers
  significant limitations, underscoring the need for a balanced and holistic
  approach to both perception and reasoning. For more teaser and testing, visit
  our project page (https://verify-eqh.pages.dev/).
paper_ai_summary: VERIFY is a benchmark designed to evaluate the visual reasoning capabilities of MLLMs by minimizing textual context and assessing reasoning paths and fidelity.
paper_ai_keywords: visual reasoning Multimodal Large Language Models (MLLMs) benchmark recognition-based skills human-annotated reasoning path visual reasoning fidelity