title: MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal
    Models
summary: IQ testing has served as a foundational methodology for evaluating human
  cognitive capabilities, deliberately decoupling assessment from linguistic
  background, language proficiency, or domain-specific knowledge to isolate core
  competencies in abstraction and reasoning. Yet, artificial intelligence
  research currently lacks systematic benchmarks to quantify these critical
  cognitive dimensions in multimodal systems. To address this critical gap, we
  propose MM-IQ, a comprehensive evaluation framework comprising 2,710
  meticulously curated test items spanning 8 distinct reasoning paradigms.
    Through systematic evaluation of leading open-source and proprietary
  multimodal models, our benchmark reveals striking limitations: even
  state-of-the-art architectures achieve only marginally superior performance to
  random chance (27.49% vs. 25% baseline accuracy). This substantial performance
  chasm highlights the inadequacy of current multimodal systems in approximating
  fundamental human reasoning capacities, underscoring the need for
  paradigm-shifting advancements to bridge this cognitive divide.
paper_authors._id: 67a1b8afe03dbbbbb51bb5c1
paper_authors.name: Huanqia Cai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-04T09:39:06.531Z
paper_authors._id: 633b99cfc9b44f5c6ac8fe03
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633b99cfc9b44f5c6ac8fe03/sFmpPlWwo07ttcWWuV1Iw.jpeg
paper_authors.fullname: huanqiacai
paper_authors.type: user
paper_authors.user: huanqia
paper_authors._id: 67a1b8afe03dbbbbb51bb5c2
paper_authors.name: Yijun Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-04T14:48:36.068Z
paper_authors._id: 645e553c3b6d85c65e8b0e54
paper_authors.avatarUrl: /avatars/1fffc6499b9d65b21a895ca96f03b781.svg
paper_authors.fullname: Steven
paper_authors.type: user
paper_authors.user: yijunyang
paper_authors._id: 67a1b8afe03dbbbbb51bb5c3
paper_authors.name: Winston Hu
paper_title: MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal
    Models
paper_summary: IQ testing has served as a foundational methodology for evaluating human
  cognitive capabilities, deliberately decoupling assessment from linguistic
  background, language proficiency, or domain-specific knowledge to isolate core
  competencies in abstraction and reasoning. Yet, artificial intelligence
  research currently lacks systematic benchmarks to quantify these critical
  cognitive dimensions in multimodal systems. To address this critical gap, we
  propose MM-IQ, a comprehensive evaluation framework comprising 2,710
  meticulously curated test items spanning 8 distinct reasoning paradigms.
    Through systematic evaluation of leading open-source and proprietary
  multimodal models, our benchmark reveals striking limitations: even
  state-of-the-art architectures achieve only marginally superior performance to
  random chance (27.49% vs. 25% baseline accuracy). This substantial performance
  chasm highlights the inadequacy of current multimodal systems in approximating
  fundamental human reasoning capacities, underscoring the need for
  paradigm-shifting advancements to bridge this cognitive divide.