title: Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with
    Minimalist Rule-Based RL
summary: Improving performance on complex tasks and enabling interpretable decision
  making in large language models (LLMs), especially for clinical applications,
  requires effective reasoning. Yet this remains challenging without supervised
  fine-tuning (SFT) on costly chain-of-thought (CoT) data distilled from
  closed-source models (e.g., GPT-4o). In this work, we present AlphaMed, the
  first medical LLM to show that reasoning capability can emerge purely through
  reinforcement learning (RL), using minimalist rule-based rewards on public
  multiple-choice QA datasets, without relying on SFT or distilled CoT data.
  AlphaMed achieves state-of-the-art results on six medical QA benchmarks,
  outperforming models trained with conventional SFT+RL pipelines. On challenging
  benchmarks (e.g., MedXpert), AlphaMed even surpasses larger or closed-source
  models such as DeepSeek-V3-671B and Claude-3.5-Sonnet. To understand the
  factors behind this success, we conduct a comprehensive data-centric analysis
  guided by three questions: (i) Can minimalist rule-based RL incentivize
  reasoning without distilled CoT supervision? (ii) How do dataset quantity and
  diversity impact reasoning? (iii) How does question difficulty shape the
  emergence and generalization of reasoning? Our findings show that dataset
  informativeness is a key driver of reasoning performance, and that minimalist
  RL on informative, multiple-choice QA data is effective at inducing reasoning
  without CoT supervision. We also observe divergent trends across benchmarks,
  underscoring limitations in current evaluation and the need for more
  challenging, reasoning-oriented medical QA benchmarks.
paper_authors._id: 6836e9b4481e0bfedacc61bc
paper_authors.name: Che Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:37:55.201Z
paper_authors._id: 631b9ff5824f2502e3557c7e
paper_authors.avatarUrl: /avatars/076043c9dba07644a570692563ef8114.svg
paper_authors.fullname: liu
paper_authors.type: user
paper_authors.user: che111
paper_authors._id: 6836e9b4481e0bfedacc61bd
paper_authors.name: Haozhe Wang
paper_authors._id: 6836e9b4481e0bfedacc61be
paper_authors.name: Jiazhen Pan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T19:14:16.399Z
paper_authors._id: 66588c8a338165aad1516756
paper_authors.avatarUrl: /avatars/c6539b4ef65f465f6f762628d6921be6.svg
paper_authors.fullname: JZPeterPan
paper_authors.type: user
paper_authors.user: JZPeterPan
paper_authors._id: 6836e9b4481e0bfedacc61bf
paper_authors.name: Zhongwei Wan
paper_authors._id: 6836e9b4481e0bfedacc61c0
paper_authors.name: Yong Dai
paper_authors._id: 6836e9b4481e0bfedacc61c1
paper_authors.name: Fangzhen Lin
paper_authors._id: 6836e9b4481e0bfedacc61c2
paper_authors.name: Wenjia Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:48:49.737Z
paper_authors._id: 67c973029d03a429bb2bbf99
paper_authors.avatarUrl: /avatars/d05246e3d1e54e13acd1959155931dac.svg
paper_authors.fullname: Bai
paper_authors.type: user
paper_authors.user: baiwenjia
paper_authors._id: 6836e9b4481e0bfedacc61c3
paper_authors.name: Daniel Rueckert
paper_authors._id: 6836e9b4481e0bfedacc61c4
paper_authors.name: Rossella Arcucci
paper_title: Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with
    Minimalist Rule-Based RL
paper_summary: Improving performance on complex tasks and enabling interpretable decision
  making in large language models (LLMs), especially for clinical applications,
  requires effective reasoning. Yet this remains challenging without supervised
  fine-tuning (SFT) on costly chain-of-thought (CoT) data distilled from
  closed-source models (e.g., GPT-4o). In this work, we present AlphaMed, the
  first medical LLM to show that reasoning capability can emerge purely through
  reinforcement learning (RL), using minimalist rule-based rewards on public
  multiple-choice QA datasets, without relying on SFT or distilled CoT data.
  AlphaMed achieves state-of-the-art results on six medical QA benchmarks,
  outperforming models trained with conventional SFT+RL pipelines. On challenging
  benchmarks (e.g., MedXpert), AlphaMed even surpasses larger or closed-source
  models such as DeepSeek-V3-671B and Claude-3.5-Sonnet. To understand the
  factors behind this success, we conduct a comprehensive data-centric analysis
  guided by three questions: (i) Can minimalist rule-based RL incentivize
  reasoning without distilled CoT supervision? (ii) How do dataset quantity and
  diversity impact reasoning? (iii) How does question difficulty shape the
  emergence and generalization of reasoning? Our findings show that dataset
  informativeness is a key driver of reasoning performance, and that minimalist
  RL on informative, multiple-choice QA data is effective at inducing reasoning
  without CoT supervision. We also observe divergent trends across benchmarks,
  underscoring limitations in current evaluation and the need for more
  challenging, reasoning-oriented medical QA benchmarks.
paper_ai_summary: AlphaMed, a medical LLM, demonstrates superior reasoning capabilities through reinforcement learning using minimalist rule-based rewards, surpassing conventionally trained models on medical QA benchmarks.
paper_ai_keywords: reinforcement learning rule-based rewards medical LLM chain-of-thought reasoning capability multiple-choice QA datasets dataset informativeness medical QA benchmarks