title: MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs
summary: Logical reasoning is a fundamental aspect of human intelligence and an
  essential capability for multimodal large language models (MLLMs). Despite the
  significant advancement in multimodal reasoning, existing benchmarks fail to
  comprehensively evaluate their reasoning abilities due to the lack of explicit
  categorization for logical reasoning types and an unclear understanding of
  reasoning. To address these issues, we introduce MME-Reasoning, a comprehensive
  benchmark designed to evaluate the reasoning ability of MLLMs, which covers all
  three types of reasoning (i.e., inductive, deductive, and abductive) in its
  questions. We carefully curate the data to ensure that each question
  effectively evaluates reasoning ability rather than perceptual skills or
  knowledge breadth, and extend the evaluation protocols to cover the evaluation
  of diverse questions. Our evaluation reveals substantial limitations of
  state-of-the-art MLLMs when subjected to holistic assessments of logical
  reasoning capabilities. Even the most advanced MLLMs show limited performance
  in comprehensive logical reasoning, with notable performance imbalances across
  reasoning types. In addition, we conducted an in-depth analysis of approaches
  such as ``thinking mode'' and Rule-based RL, which are commonly believed to
  enhance reasoning abilities. These findings highlight the critical limitations
  and performance imbalances of current MLLMs in diverse logical reasoning
  scenarios, providing comprehensive and systematic insights into the
  understanding and evaluation of reasoning capabilities.
paper_authors._id: 6836799db9b35de1c4a90d73
paper_authors.name: Jiakang Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:10:39.171Z
paper_authors._id: 64a3d1ddb3239f3e3892b24b
paper_authors.avatarUrl: /avatars/7ce585f5fc1d077fb1d70cc18c4da2c1.svg
paper_authors.fullname: Jiakang Yuan
paper_authors.type: user
paper_authors.user: JiakangYuan
paper_authors._id: 6836799db9b35de1c4a90d74
paper_authors.name: Tianshuo Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:10:47.745Z
paper_authors._id: 6538dd471ad9b3ba7c2df861
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6538dd471ad9b3ba7c2df861/MbEa7KHAK6u7PRb7WiPUC.jpeg
paper_authors.fullname: Tianshuo Peng
paper_authors.type: user
paper_authors.user: Potentialts
paper_authors._id: 6836799db9b35de1c4a90d75
paper_authors.name: Yilei Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:11:01.744Z
paper_authors._id: 66838e043f885cb431c48fd9
paper_authors.avatarUrl: /avatars/b14e238dabb99a692ce2189f1c7ec9c7.svg
paper_authors.fullname: JiangYilei
paper_authors.type: user
paper_authors.user: Yilei-Jiang
paper_authors._id: 6836799db9b35de1c4a90d76
paper_authors.name: Yiting Lu
paper_authors._id: 6836799db9b35de1c4a90d77
paper_authors.name: Renrui Zhang
paper_authors._id: 6836799db9b35de1c4a90d78
paper_authors.name: Kaituo Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:12:45.346Z
paper_authors._id: 67079840a9bcb7459b8d2a46
paper_authors.avatarUrl: /avatars/32466863c5554f20cb2775b138832ac3.svg
paper_authors.fullname: Kaituo Feng
paper_authors.type: user
paper_authors.user: KaituoFeng
paper_authors._id: 6836799db9b35de1c4a90d79
paper_authors.name: Chaoyou Fu
paper_authors._id: 6836799db9b35de1c4a90d7a
paper_authors.name: Tao Chen
paper_authors._id: 6836799db9b35de1c4a90d7b
paper_authors.name: Lei Bai
paper_authors._id: 6836799db9b35de1c4a90d7c
paper_authors.name: Bo Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:13:12.150Z
paper_authors._id: 643dfd235aafbdca3a5792c0
paper_authors.avatarUrl: /avatars/ce8553cf5936012c692e08054ee27937.svg
paper_authors.fullname: Bo Zhang
paper_authors.type: user
paper_authors.user: BoZhang
paper_authors._id: 6836799db9b35de1c4a90d7d
paper_authors.name: Xiangyu Yue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:12:59.396Z
paper_authors._id: 666a8f24e2990b0cb16b7bf9
paper_authors.avatarUrl: /avatars/fcbaf8f1e3e53a2a4a819b7cb2c53aa4.svg
paper_authors.fullname: Xiangyu Yue
paper_authors.type: user
paper_authors.user: xyyue
paper_title: MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs
paper_summary: Logical reasoning is a fundamental aspect of human intelligence and an
  essential capability for multimodal large language models (MLLMs). Despite the
  significant advancement in multimodal reasoning, existing benchmarks fail to
  comprehensively evaluate their reasoning abilities due to the lack of explicit
  categorization for logical reasoning types and an unclear understanding of
  reasoning. To address these issues, we introduce MME-Reasoning, a comprehensive
  benchmark designed to evaluate the reasoning ability of MLLMs, which covers all
  three types of reasoning (i.e., inductive, deductive, and abductive) in its
  questions. We carefully curate the data to ensure that each question
  effectively evaluates reasoning ability rather than perceptual skills or
  knowledge breadth, and extend the evaluation protocols to cover the evaluation
  of diverse questions. Our evaluation reveals substantial limitations of
  state-of-the-art MLLMs when subjected to holistic assessments of logical
  reasoning capabilities. Even the most advanced MLLMs show limited performance
  in comprehensive logical reasoning, with notable performance imbalances across
  reasoning types. In addition, we conducted an in-depth analysis of approaches
  such as ``thinking mode'' and Rule-based RL, which are commonly believed to
  enhance reasoning abilities. These findings highlight the critical limitations
  and performance imbalances of current MLLMs in diverse logical reasoning
  scenarios, providing comprehensive and systematic insights into the
  understanding and evaluation of reasoning capabilities.
paper_ai_summary: MME-Reasoning evaluates the logical reasoning capabilities of multimodal large language models, revealing significant limitations and performance imbalances across inductive, deductive, and abductive reasoning types.
paper_ai_keywords: multimodal large language models MME-Reasoning logical reasoning inductive reasoning deductive reasoning abductive reasoning reasoning ability thinking mode Rule-based RL