HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
The breakthrough of OpenAI o1 highlights the potential of enhancing reasoning
to improve LLM. Yet, most research in reasoning has focused on mathematical
tasks, leaving domains like medicine underexplored. The medical domain, though
distinct from mathematics, also demands robust reasoning to provide reliable
answers, given the high standards of healthcare. However, verifying medical
reasoning is challenging, unlike those in mathematics. To address this, we
propose verifiable medical problems with a medical verifier to check the
correctness of model outputs. This verifiable nature enables advancements in
medical reasoning through a two-stage approach: (1) using the verifier to guide
the search for a complex reasoning trajectory for fine-tuning LLMs, (2)
applying reinforcement learning (RL) with verifier-based rewards to enhance
complex reasoning further. Finally, we introduce HuatuoGPT-o1, a medical LLM
capable of complex reasoning, which outperforms general and medical-specific
baselines using only 40K verifiable problems. Experiments show complex
reasoning improves medical problem-solving and benefits more from RL. We hope
our approach inspires advancements in reasoning across medical and other
specialized domains.
677209448e0ed7713b183674
Junying Chen
admin_assigned
2024-12-30T20:04:26.351Z
64097dd1b6a334f53e2b3e4c
/avatars/18d036aab5e096054a8706bc78027126.svg
Junying Chen
user
jymcc
677209448e0ed7713b183675
Zhenyang Cai
admin_assigned
2024-12-30T19:57:15.576Z
64f1a34f2c5c8b767916447e
https://cdn-avatars.huggingface.co/v1/production/uploads/64f1a34f2c5c8b767916447e/uak2CsMAnxW8q4dwyAOBN.jpeg
Zhenyang Cai
user
Eric3200
677209448e0ed7713b183676
Ke Ji
677209448e0ed7713b183677
Xidong Wang
admin_assigned
2024-12-30T19:57:22.852Z
640ed3e9f2d7c41a1e9a9fde
https://cdn-avatars.huggingface.co/v1/production/uploads/1678693486092-640ed3e9f2d7c41a1e9a9fde.jpeg
Xidong Wang
user
Xidong
677209448e0ed7713b183678
Wanlong Liu
admin_assigned
2024-12-30T20:03:23.956Z
64eb333e6878d90b031fa5c5
/avatars/a0d875b49d1c56be88f34854647306da.svg
Wanlong Liu
user
lwl-uestc
677209448e0ed7713b183679
Rongsheng Wang
claimed_verified
2024-12-30T19:30:46.521Z
63ca949b04c979828315389d
https://cdn-avatars.huggingface.co/v1/production/uploads/63ca949b04c979828315389d/HS5xWNAYjjHeyAAwWJ11l.jpeg
wangrongsheng
user
wangrongsheng
677209448e0ed7713b18367a
Jianye Hou
677209448e0ed7713b18367b
Benyou Wang
admin_assigned
2024-12-30T20:03:08.303Z
637c6703ca8542a0ba900ccb
/avatars/288ed63a1efa566c3f01e850c6ba5dd5.svg
Wang
user
Benyou
HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
The breakthrough of OpenAI o1 highlights the potential of enhancing reasoning
to improve LLM. Yet, most research in reasoning has focused on mathematical
tasks, leaving domains like medicine underexplored. The medical domain, though
distinct from mathematics, also demands robust reasoning to provide reliable
answers, given the high standards of healthcare. However, verifying medical
reasoning is challenging, unlike those in mathematics. To address this, we
propose verifiable medical problems with a medical verifier to check the
correctness of model outputs. This verifiable nature enables advancements in
medical reasoning through a two-stage approach: (1) using the verifier to guide
the search for a complex reasoning trajectory for fine-tuning LLMs, (2)
applying reinforcement learning (RL) with verifier-based rewards to enhance
complex reasoning further. Finally, we introduce HuatuoGPT-o1, a medical LLM
capable of complex reasoning, which outperforms general and medical-specific
baselines using only 40K verifiable problems. Experiments show complex
reasoning improves medical problem-solving and benefits more from RL. We hope
our approach inspires advancements in reasoning across medical and other
specialized domains.
The proposed approach enhances medical reasoning in large language models using verifiable problems, guided search, and reinforcement learning to achieve superior performance in medical problem-solving.
LLM reasoning OpenAI medical verifier fine-tuning reinforcement learning RL HuatuoGPT-o1