title: MedXpertQA: Benchmarking Expert-Level Medical Reasoning and
    Understanding
summary: We introduce MedXpertQA, a highly challenging and comprehensive benchmark to
  evaluate expert-level medical knowledge and advanced reasoning. MedXpertQA
  includes 4,460 questions spanning 17 specialties and 11 body systems. It
  includes two subsets, Text for text evaluation and MM for multimodal
  evaluation. Notably, MM introduces expert-level exam questions with diverse
  images and rich clinical information, including patient records and examination
  results, setting it apart from traditional medical multimodal benchmarks with
  simple QA pairs generated from image captions. MedXpertQA applies rigorous
  filtering and augmentation to address the insufficient difficulty of existing
  benchmarks like MedQA, and incorporates specialty board questions to improve
  clinical relevance and comprehensiveness. We perform data synthesis to mitigate
  data leakage risk and conduct multiple rounds of expert reviews to ensure
  accuracy and reliability. We evaluate 16 leading models on MedXpertQA.
  Moreover, medicine is deeply connected to real-world decision-making, providing
  a rich and representative setting for assessing reasoning abilities beyond
  mathematics and code. To this end, we develop a reasoning-oriented subset to
  facilitate the assessment of o1-like models.
paper_authors._id: 679c5b0034f5df4416915177
paper_authors.name: Yuxin Zuo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-23T15:57:12.013Z
paper_authors._id: 622474f38dc6b0b64f5e903d
paper_authors.avatarUrl: /avatars/d6b60a014277a8ec7d564163c5f644aa.svg
paper_authors.fullname: Yuxin Zuo
paper_authors.type: user
paper_authors.user: yuxinzuo
paper_authors._id: 679c5b0034f5df4416915178
paper_authors.name: Shang Qu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-31T08:35:48.269Z
paper_authors._id: 65597738deee83130a1301d5
paper_authors.avatarUrl: /avatars/9bcc40aebe4db079927675d95c00463c.svg
paper_authors.fullname: Shang (Lindsay) Qu
paper_authors.type: user
paper_authors.user: lindsay-qu
paper_authors._id: 679c5b0034f5df4416915179
paper_authors.name: Yifei Li
paper_authors._id: 679c5b0034f5df441691517a
paper_authors.name: Zhangren Chen
paper_authors._id: 679c5b0034f5df441691517b
paper_authors.name: Xuekai Zhu
paper_authors._id: 679c5b0034f5df441691517c
paper_authors.name: Ermo Hua
paper_authors._id: 679c5b0034f5df441691517d
paper_authors.name: Kaiyan Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-11T15:33:58.392Z
paper_authors._id: 60bc94cd85a3ab33829b6211
paper_authors.avatarUrl: /avatars/b57d36c7577fbbb42ea5b963eef4144a.svg
paper_authors.fullname: Kaiyan Zhang
paper_authors.type: user
paper_authors.user: iseesaw
paper_authors._id: 679c5b0034f5df441691517e
paper_authors.name: Ning Ding
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-31T09:50:45.999Z
paper_authors._id: 60cf4bcb1ce3775ebb86e5d5
paper_authors.avatarUrl: /avatars/12bcd18d215abf91f297f93007733148.svg
paper_authors.fullname: Ning Ding
paper_authors.type: user
paper_authors.user: stingning
paper_authors._id: 679c5b0034f5df441691517f
paper_authors.name: Bowen Zhou
paper_title: MedXpertQA: Benchmarking Expert-Level Medical Reasoning and
    Understanding
paper_summary: We introduce MedXpertQA, a highly challenging and comprehensive benchmark to
  evaluate expert-level medical knowledge and advanced reasoning. MedXpertQA
  includes 4,460 questions spanning 17 specialties and 11 body systems. It
  includes two subsets, Text for text evaluation and MM for multimodal
  evaluation. Notably, MM introduces expert-level exam questions with diverse
  images and rich clinical information, including patient records and examination
  results, setting it apart from traditional medical multimodal benchmarks with
  simple QA pairs generated from image captions. MedXpertQA applies rigorous
  filtering and augmentation to address the insufficient difficulty of existing
  benchmarks like MedQA, and incorporates specialty board questions to improve
  clinical relevance and comprehensiveness. We perform data synthesis to mitigate
  data leakage risk and conduct multiple rounds of expert reviews to ensure
  accuracy and reliability. We evaluate 16 leading models on MedXpertQA.
  Moreover, medicine is deeply connected to real-world decision-making, providing
  a rich and representative setting for assessing reasoning abilities beyond
  mathematics and code. To this end, we develop a reasoning-oriented subset to
  facilitate the assessment of o1-like models.
paper_ai_summary: MedXpertQA is a benchmark for evaluating medical expertise and reasoning with a focus on advanced clinical knowledge and diverse data, including images and patient records.
paper_ai_keywords: benchmark MedXpertQA text evaluation multimodal evaluation expert-level questions clinical information data synthesis expert reviews reasoning-oriented subset