MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in
  Multimodal Large Language Models
Multimodal reasoning, which integrates language and visual cues into problem
solving and decision making, is a fundamental aspect of human intelligence and
a crucial step toward artificial general intelligence. However, the evaluation
of multimodal reasoning capabilities in Multimodal Large Language Models
(MLLMs) remains inadequate. Most existing reasoning benchmarks are constrained
by limited data size, narrow domain coverage, and unstructured knowledge
distribution. To close these gaps, we introduce MDK12-Bench, a
multi-disciplinary benchmark assessing the reasoning capabilities of MLLMs via
real-world K-12 examinations. Spanning six disciplines (math, physics,
chemistry, biology, geography, and information science), our benchmark
comprises 140K reasoning instances across diverse difficulty levels from
primary school to 12th grade. It features 6,827 instance-level knowledge point
annotations based on a well-organized knowledge structure, detailed answer
explanations, difficulty labels and cross-year partitions, providing a robust
platform for comprehensive evaluation. Additionally, we present a novel dynamic
evaluation framework to mitigate data contamination issues by bootstrapping
question forms, question types, and image styles during evaluation. Extensive
experiment on MDK12-Bench reveals the significant limitation of current MLLMs
in multimodal reasoning. The findings on our benchmark provide insights into
the development of the next-generation models. Our data and codes are available
at https://github.com/LanceZPF/MDK12.
67fe419a7161674011524204
Pengfei Zhou
67fe419a7161674011524205
Fanrui Zhang
67fe419a7161674011524206
Xiaopeng Peng
67fe419a7161674011524207
Zhaopan Xu
67fe419a7161674011524208
Jiaxin Ai
67fe419a7161674011524209
Yansheng Qiu
67fe419a716167401152420a
Chuanhao Li
claimed_verified
2025-07-17T13:55:08.745Z
674ea59a8f2e7614a6c72f26
/avatars/86fd4c6d7d33435de49b35659bf65265.svg
Chuanhao
user
ChuanhaoLi
67fe419a716167401152420b
Zhen Li
67fe419a716167401152420c
Ming Li
67fe419a716167401152420d
Yukang Feng
67fe419a716167401152420e
Jianwen Sun
67fe419a716167401152420f
Haoquan Zhang
67fe419a7161674011524210
Zizhen Li
67fe419a7161674011524211
Xiaofeng Mao
67fe419a7161674011524212
Wangbo Zhao
67fe419a7161674011524213
Kai Wang
67fe419a7161674011524214
Xiaojun Chang
67fe419a7161674011524215
Wenqi Shao
67fe419a7161674011524216
Yang You
67fe419a7161674011524217
Kaipeng Zhang
claimed_verified
2025-06-02T07:50:54.408Z
65f1713552c38a91e0a445e8
/avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
kaipeng
user
kpzhang996
MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in
  Multimodal Large Language Models
Multimodal reasoning, which integrates language and visual cues into problem
solving and decision making, is a fundamental aspect of human intelligence and
a crucial step toward artificial general intelligence. However, the evaluation
of multimodal reasoning capabilities in Multimodal Large Language Models
(MLLMs) remains inadequate. Most existing reasoning benchmarks are constrained
by limited data size, narrow domain coverage, and unstructured knowledge
distribution. To close these gaps, we introduce MDK12-Bench, a
multi-disciplinary benchmark assessing the reasoning capabilities of MLLMs via
real-world K-12 examinations. Spanning six disciplines (math, physics,
chemistry, biology, geography, and information science), our benchmark
comprises 140K reasoning instances across diverse difficulty levels from
primary school to 12th grade. It features 6,827 instance-level knowledge point
annotations based on a well-organized knowledge structure, detailed answer
explanations, difficulty labels and cross-year partitions, providing a robust
platform for comprehensive evaluation. Additionally, we present a novel dynamic
evaluation framework to mitigate data contamination issues by bootstrapping
question forms, question types, and image styles during evaluation. Extensive
experiment on MDK12-Bench reveals the significant limitation of current MLLMs
in multimodal reasoning. The findings on our benchmark provide insights into
the development of the next-generation models. Our data and codes are available
at https://github.com/LanceZPF/MDK12.
MDK12-Bench evaluates multimodal reasoning capabilities of MLLMs using diverse real-world educational tests across multiple disciplines.
multimodal reasoning Multimodal Large Language Models MDK12-Bench reasoning instances knowledge point annotations dynamic evaluation framework