title: O1 Replication Journey -- Part 3: Inference-time Scaling for Medical
    Reasoning
summary: Building upon our previous investigations of O1 replication (Part 1: Journey
  Learning [Qin et al., 2024] and Part 2: Distillation [Huang et al., 2024]),
  this work explores the potential of inference-time scaling in large language
  models (LLMs) for medical reasoning tasks, ranging from diagnostic
  decision-making to treatment planning. Through extensive experiments on medical
  benchmarks of varying complexity (MedQA, Medbullets, and JAMA Clinical
  Challenges), our investigation reveals several key insights: (1) Increasing
  inference time does lead to improved performance. With a modest training set of
  500 samples, our model yields substantial performance improvements of 6%-11%.
  (2) Task complexity directly correlates with the required length of reasoning
  chains, confirming the necessity of extended thought processes for challenging
  problems. (3) The differential diagnoses generated by our model adhere to the
  principles of the hypothetico-deductive method, producing a list of potential
  conditions that may explain a patient's symptoms and systematically narrowing
  these possibilities by evaluating the evidence. These findings demonstrate the
  promising synergy between inference-time scaling and journey learning in
  advancing LLMs' real-world clinical reasoning capabilities.
paper_authors._id: 6785e65a7d58f2945af8034d
paper_authors.name: Zhongzhen Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-14T09:10:03.906Z
paper_authors._id: 620a0a2e8d5e5dfed284bc41
paper_authors.avatarUrl: /avatars/ca2ddad577227103cdf263a4b7817b22.svg
paper_authors.fullname: huangzhongzhen
paper_authors.type: user
paper_authors.user: zongzi
paper_authors._id: 6785e65a7d58f2945af8034e
paper_authors.name: Gui Geng
paper_authors._id: 6785e65a7d58f2945af8034f
paper_authors.name: Shengyi Hua
paper_authors._id: 6785e65a7d58f2945af80350
paper_authors.name: Zhen Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-14T09:12:15.207Z
paper_authors._id: 643581a4f3b08e267d990499
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643581a4f3b08e267d990499/KRhB-48W4IPuB0bX16Ahj.png
paper_authors.fullname: Zhen Huang
paper_authors.type: user
paper_authors.user: ZhenHuang
paper_authors._id: 6785e65a7d58f2945af80351
paper_authors.name: Haoyang Zou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-14T09:12:22.954Z
paper_authors._id: 65011b18af1e4021439af578
paper_authors.avatarUrl: /avatars/ae1632c800ceeb43aa223e432f70bc6d.svg
paper_authors.fullname: Haoyang Zou
paper_authors.type: user
paper_authors.user: alanyoung058
paper_authors._id: 6785e65a7d58f2945af80352
paper_authors.name: Shaoting Zhang
paper_authors._id: 6785e65a7d58f2945af80353
paper_authors.name: Pengfei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-14T09:12:46.582Z
paper_authors._id: 6144a0c4ff1146bbd84d9865
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
paper_authors.fullname: Pengfei Liu
paper_authors.type: user
paper_authors.user: Pengfei
paper_authors._id: 6785e65a7d58f2945af80354
paper_authors.name: Xiaofan Zhang
paper_title: O1 Replication Journey -- Part 3: Inference-time Scaling for Medical
    Reasoning
paper_summary: Building upon our previous investigations of O1 replication (Part 1: Journey
  Learning [Qin et al., 2024] and Part 2: Distillation [Huang et al., 2024]),
  this work explores the potential of inference-time scaling in large language
  models (LLMs) for medical reasoning tasks, ranging from diagnostic
  decision-making to treatment planning. Through extensive experiments on medical
  benchmarks of varying complexity (MedQA, Medbullets, and JAMA Clinical
  Challenges), our investigation reveals several key insights: (1) Increasing
  inference time does lead to improved performance. With a modest training set of
  500 samples, our model yields substantial performance improvements of 6%-11%.
  (2) Task complexity directly correlates with the required length of reasoning
  chains, confirming the necessity of extended thought processes for challenging
  problems. (3) The differential diagnoses generated by our model adhere to the
  principles of the hypothetico-deductive method, producing a list of potential
  conditions that may explain a patient's symptoms and systematically narrowing
  these possibilities by evaluating the evidence. These findings demonstrate the
  promising synergy between inference-time scaling and journey learning in
  advancing LLMs' real-world clinical reasoning capabilities.
paper_ai_summary: Inference-time scaling in large language models enhances medical reasoning performance by increasing the length of reasoning chains and adhering to the hypothetico-deductive method in generating differential diagnoses.
paper_ai_keywords: inference-time scaling large language models medical reasoning diagnostic decision-making treatment planning MedQA Medbullets JAMA Clinical Challenges reasoning chains hypothetico-deductive method differential diagnoses