title: HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
summary: The breakthrough of OpenAI o1 highlights the potential of enhancing reasoning
  to improve LLM. Yet, most research in reasoning has focused on mathematical
  tasks, leaving domains like medicine underexplored. The medical domain, though
  distinct from mathematics, also demands robust reasoning to provide reliable
  answers, given the high standards of healthcare. However, verifying medical
  reasoning is challenging, unlike those in mathematics. To address this, we
  propose verifiable medical problems with a medical verifier to check the
  correctness of model outputs. This verifiable nature enables advancements in
  medical reasoning through a two-stage approach: (1) using the verifier to guide
  the search for a complex reasoning trajectory for fine-tuning LLMs, (2)
  applying reinforcement learning (RL) with verifier-based rewards to enhance
  complex reasoning further. Finally, we introduce HuatuoGPT-o1, a medical LLM
  capable of complex reasoning, which outperforms general and medical-specific
  baselines using only 40K verifiable problems. Experiments show complex
  reasoning improves medical problem-solving and benefits more from RL. We hope
  our approach inspires advancements in reasoning across medical and other
  specialized domains.
paper_authors._id: 677209448e0ed7713b183674
paper_authors.name: Junying Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T20:04:26.351Z
paper_authors._id: 64097dd1b6a334f53e2b3e4c
paper_authors.avatarUrl: /avatars/18d036aab5e096054a8706bc78027126.svg
paper_authors.fullname: Junying Chen
paper_authors.type: user
paper_authors.user: jymcc
paper_authors._id: 677209448e0ed7713b183675
paper_authors.name: Zhenyang Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T19:57:15.576Z
paper_authors._id: 64f1a34f2c5c8b767916447e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64f1a34f2c5c8b767916447e/uak2CsMAnxW8q4dwyAOBN.jpeg
paper_authors.fullname: Zhenyang Cai
paper_authors.type: user
paper_authors.user: Eric3200
paper_authors._id: 677209448e0ed7713b183676
paper_authors.name: Ke Ji
paper_authors._id: 677209448e0ed7713b183677
paper_authors.name: Xidong Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T19:57:22.852Z
paper_authors._id: 640ed3e9f2d7c41a1e9a9fde
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678693486092-640ed3e9f2d7c41a1e9a9fde.jpeg
paper_authors.fullname: Xidong Wang
paper_authors.type: user
paper_authors.user: Xidong
paper_authors._id: 677209448e0ed7713b183678
paper_authors.name: Wanlong Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T20:03:23.956Z
paper_authors._id: 64eb333e6878d90b031fa5c5
paper_authors.avatarUrl: /avatars/a0d875b49d1c56be88f34854647306da.svg
paper_authors.fullname: Wanlong Liu
paper_authors.type: user
paper_authors.user: lwl-uestc
paper_authors._id: 677209448e0ed7713b183679
paper_authors.name: Rongsheng Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:30:46.521Z
paper_authors._id: 63ca949b04c979828315389d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63ca949b04c979828315389d/HS5xWNAYjjHeyAAwWJ11l.jpeg
paper_authors.fullname: wangrongsheng
paper_authors.type: user
paper_authors.user: wangrongsheng
paper_authors._id: 677209448e0ed7713b18367a
paper_authors.name: Jianye Hou
paper_authors._id: 677209448e0ed7713b18367b
paper_authors.name: Benyou Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T20:03:08.303Z
paper_authors._id: 637c6703ca8542a0ba900ccb
paper_authors.avatarUrl: /avatars/288ed63a1efa566c3f01e850c6ba5dd5.svg
paper_authors.fullname: Wang
paper_authors.type: user
paper_authors.user: Benyou
paper_title: HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
paper_summary: The breakthrough of OpenAI o1 highlights the potential of enhancing reasoning
  to improve LLM. Yet, most research in reasoning has focused on mathematical
  tasks, leaving domains like medicine underexplored. The medical domain, though
  distinct from mathematics, also demands robust reasoning to provide reliable
  answers, given the high standards of healthcare. However, verifying medical
  reasoning is challenging, unlike those in mathematics. To address this, we
  propose verifiable medical problems with a medical verifier to check the
  correctness of model outputs. This verifiable nature enables advancements in
  medical reasoning through a two-stage approach: (1) using the verifier to guide
  the search for a complex reasoning trajectory for fine-tuning LLMs, (2)
  applying reinforcement learning (RL) with verifier-based rewards to enhance
  complex reasoning further. Finally, we introduce HuatuoGPT-o1, a medical LLM
  capable of complex reasoning, which outperforms general and medical-specific
  baselines using only 40K verifiable problems. Experiments show complex
  reasoning improves medical problem-solving and benefits more from RL. We hope
  our approach inspires advancements in reasoning across medical and other
  specialized domains.
paper_ai_summary: The proposed approach enhances medical reasoning in large language models using verifiable problems, guided search, and reinforcement learning to achieve superior performance in medical problem-solving.
paper_ai_keywords: LLM reasoning OpenAI medical verifier fine-tuning reinforcement learning RL HuatuoGPT-o1