MedXpertQA: Benchmarking Expert-Level Medical Reasoning and
  Understanding
We introduce MedXpertQA, a highly challenging and comprehensive benchmark to
evaluate expert-level medical knowledge and advanced reasoning. MedXpertQA
includes 4,460 questions spanning 17 specialties and 11 body systems. It
includes two subsets, Text for text evaluation and MM for multimodal
evaluation. Notably, MM introduces expert-level exam questions with diverse
images and rich clinical information, including patient records and examination
results, setting it apart from traditional medical multimodal benchmarks with
simple QA pairs generated from image captions. MedXpertQA applies rigorous
filtering and augmentation to address the insufficient difficulty of existing
benchmarks like MedQA, and incorporates specialty board questions to improve
clinical relevance and comprehensiveness. We perform data synthesis to mitigate
data leakage risk and conduct multiple rounds of expert reviews to ensure
accuracy and reliability. We evaluate 16 leading models on MedXpertQA.
Moreover, medicine is deeply connected to real-world decision-making, providing
a rich and representative setting for assessing reasoning abilities beyond
mathematics and code. To this end, we develop a reasoning-oriented subset to
facilitate the assessment of o1-like models.
679c5b0034f5df4416915177
Yuxin Zuo
claimed_verified
2025-04-23T15:57:12.013Z
622474f38dc6b0b64f5e903d
/avatars/d6b60a014277a8ec7d564163c5f644aa.svg
Yuxin Zuo
user
yuxinzuo
679c5b0034f5df4416915178
Shang Qu
claimed_verified
2025-01-31T08:35:48.269Z
65597738deee83130a1301d5
/avatars/9bcc40aebe4db079927675d95c00463c.svg
Shang (Lindsay) Qu
user
lindsay-qu
679c5b0034f5df4416915179
Yifei Li
679c5b0034f5df441691517a
Zhangren Chen
679c5b0034f5df441691517b
Xuekai Zhu
679c5b0034f5df441691517c
Ermo Hua
679c5b0034f5df441691517d
Kaiyan Zhang
claimed_verified
2025-02-11T15:33:58.392Z
60bc94cd85a3ab33829b6211
/avatars/b57d36c7577fbbb42ea5b963eef4144a.svg
Kaiyan Zhang
user
iseesaw
679c5b0034f5df441691517e
Ning Ding
claimed_verified
2025-01-31T09:50:45.999Z
60cf4bcb1ce3775ebb86e5d5
/avatars/12bcd18d215abf91f297f93007733148.svg
Ning Ding
user
stingning
679c5b0034f5df441691517f
Bowen Zhou
MedXpertQA: Benchmarking Expert-Level Medical Reasoning and
  Understanding
We introduce MedXpertQA, a highly challenging and comprehensive benchmark to
evaluate expert-level medical knowledge and advanced reasoning. MedXpertQA
includes 4,460 questions spanning 17 specialties and 11 body systems. It
includes two subsets, Text for text evaluation and MM for multimodal
evaluation. Notably, MM introduces expert-level exam questions with diverse
images and rich clinical information, including patient records and examination
results, setting it apart from traditional medical multimodal benchmarks with
simple QA pairs generated from image captions. MedXpertQA applies rigorous
filtering and augmentation to address the insufficient difficulty of existing
benchmarks like MedQA, and incorporates specialty board questions to improve
clinical relevance and comprehensiveness. We perform data synthesis to mitigate
data leakage risk and conduct multiple rounds of expert reviews to ensure
accuracy and reliability. We evaluate 16 leading models on MedXpertQA.
Moreover, medicine is deeply connected to real-world decision-making, providing
a rich and representative setting for assessing reasoning abilities beyond
mathematics and code. To this end, we develop a reasoning-oriented subset to
facilitate the assessment of o1-like models.
MedXpertQA is a benchmark for evaluating medical expertise and reasoning with a focus on advanced clinical knowledge and diverse data, including images and patient records.
benchmark MedXpertQA text evaluation multimodal evaluation expert-level questions clinical information data synthesis expert reviews reasoning-oriented subset