title: MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in
    Multimodal Large Language Models
summary: Multimodal reasoning, which integrates language and visual cues into problem
  solving and decision making, is a fundamental aspect of human intelligence and
  a crucial step toward artificial general intelligence. However, the evaluation
  of multimodal reasoning capabilities in Multimodal Large Language Models
  (MLLMs) remains inadequate. Most existing reasoning benchmarks are constrained
  by limited data size, narrow domain coverage, and unstructured knowledge
  distribution. To close these gaps, we introduce MDK12-Bench, a
  multi-disciplinary benchmark assessing the reasoning capabilities of MLLMs via
  real-world K-12 examinations. Spanning six disciplines (math, physics,
  chemistry, biology, geography, and information science), our benchmark
  comprises 140K reasoning instances across diverse difficulty levels from
  primary school to 12th grade. It features 6,827 instance-level knowledge point
  annotations based on a well-organized knowledge structure, detailed answer
  explanations, difficulty labels and cross-year partitions, providing a robust
  platform for comprehensive evaluation. Additionally, we present a novel dynamic
  evaluation framework to mitigate data contamination issues by bootstrapping
  question forms, question types, and image styles during evaluation. Extensive
  experiment on MDK12-Bench reveals the significant limitation of current MLLMs
  in multimodal reasoning. The findings on our benchmark provide insights into
  the development of the next-generation models. Our data and codes are available
  at https://github.com/LanceZPF/MDK12.
paper_authors._id: 67fe419a7161674011524204
paper_authors.name: Pengfei Zhou
paper_authors._id: 67fe419a7161674011524205
paper_authors.name: Fanrui Zhang
paper_authors._id: 67fe419a7161674011524206
paper_authors.name: Xiaopeng Peng
paper_authors._id: 67fe419a7161674011524207
paper_authors.name: Zhaopan Xu
paper_authors._id: 67fe419a7161674011524208
paper_authors.name: Jiaxin Ai
paper_authors._id: 67fe419a7161674011524209
paper_authors.name: Yansheng Qiu
paper_authors._id: 67fe419a716167401152420a
paper_authors.name: Chuanhao Li
paper_authors._id: 67fe419a716167401152420b
paper_authors.name: Zhen Li
paper_authors._id: 67fe419a716167401152420c
paper_authors.name: Ming Li
paper_authors._id: 67fe419a716167401152420d
paper_authors.name: Yukang Feng
paper_authors._id: 67fe419a716167401152420e
paper_authors.name: Jianwen Sun
paper_authors._id: 67fe419a716167401152420f
paper_authors.name: Haoquan Zhang
paper_authors._id: 67fe419a7161674011524210
paper_authors.name: Zizhen Li
paper_authors._id: 67fe419a7161674011524211
paper_authors.name: Xiaofeng Mao
paper_authors._id: 67fe419a7161674011524212
paper_authors.name: Wangbo Zhao
paper_authors._id: 67fe419a7161674011524213
paper_authors.name: Kai Wang
paper_authors._id: 67fe419a7161674011524214
paper_authors.name: Xiaojun Chang
paper_authors._id: 67fe419a7161674011524215
paper_authors.name: Wenqi Shao
paper_authors._id: 67fe419a7161674011524216
paper_authors.name: Yang You
paper_authors._id: 67fe419a7161674011524217
paper_authors.name: Kaipeng Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:50:54.408Z
paper_authors._id: 65f1713552c38a91e0a445e8
paper_authors.avatarUrl: /avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
paper_authors.fullname: kaipeng
paper_authors.type: user
paper_authors.user: kpzhang996
paper_title: MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in
    Multimodal Large Language Models
paper_summary: Multimodal reasoning, which integrates language and visual cues into problem
  solving and decision making, is a fundamental aspect of human intelligence and
  a crucial step toward artificial general intelligence. However, the evaluation
  of multimodal reasoning capabilities in Multimodal Large Language Models
  (MLLMs) remains inadequate. Most existing reasoning benchmarks are constrained
  by limited data size, narrow domain coverage, and unstructured knowledge
  distribution. To close these gaps, we introduce MDK12-Bench, a
  multi-disciplinary benchmark assessing the reasoning capabilities of MLLMs via
  real-world K-12 examinations. Spanning six disciplines (math, physics,
  chemistry, biology, geography, and information science), our benchmark
  comprises 140K reasoning instances across diverse difficulty levels from
  primary school to 12th grade. It features 6,827 instance-level knowledge point
  annotations based on a well-organized knowledge structure, detailed answer
  explanations, difficulty labels and cross-year partitions, providing a robust
  platform for comprehensive evaluation. Additionally, we present a novel dynamic
  evaluation framework to mitigate data contamination issues by bootstrapping
  question forms, question types, and image styles during evaluation. Extensive
  experiment on MDK12-Bench reveals the significant limitation of current MLLMs
  in multimodal reasoning. The findings on our benchmark provide insights into
  the development of the next-generation models. Our data and codes are available
  at https://github.com/LanceZPF/MDK12.
paper_ai_summary: MDK12-Bench evaluates multimodal reasoning capabilities of MLLMs using diverse real-world educational tests across multiple disciplines.
paper_ai_keywords: multimodal reasoning Multimodal Large Language Models MDK12-Bench reasoning instances knowledge point annotations dynamic evaluation framework