title: Adapting While Learning: Grounding LLMs for Scientific Problems with
    Intelligent Tool Usage Adaptation
summary: Large Language Models (LLMs) demonstrate promising capabilities in solving
  simple scientific problems but often produce hallucinations for complex ones.
  While integrating LLMs with tools can increase reliability, this approach
  typically results in over-reliance on tools, diminishing the model's ability to
  solve simple problems through basic reasoning. In contrast, human experts first
  assess problem complexity using domain knowledge before choosing an appropriate
  solution approach. Inspired by this human problem-solving process, we propose a
  novel two-component fine-tuning method. In the first component World Knowledge
  Distillation (WKD), LLMs learn directly from solutions generated using tool's
  information to internalize domain knowledge. In the second component Tool Usage
  Adaptation (TUA), we partition problems into easy and hard categories based on
  the model's direct answering accuracy. While maintaining the same alignment
  target for easy problems as in WKD, we train the model to intelligently switch
  to tool usage for more challenging problems. We validate our method on six
  scientific benchmark datasets, spanning mathematics, climate science and
  epidemiology. On average, our models demonstrate a 28.18% improvement in answer
  accuracy and a 13.89% increase in tool usage precision across all datasets,
  surpassing state-of-the-art models including GPT-4o and Claude-3.5.
paper_authors._id: 67282af8fab7cbfa93389808
paper_authors.name: Bohan Lyu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-04T09:36:08.498Z
paper_authors._id: 650267e7e751d03da933a24a
paper_authors.avatarUrl: /avatars/f047a047d1de304cd97027463541bdf3.svg
paper_authors.fullname: Bohan22
paper_authors.type: user
paper_authors.user: Bohan22
paper_authors._id: 67282af8fab7cbfa93389809
paper_authors.name: Yadi Cao
paper_authors._id: 67282af8fab7cbfa9338980a
paper_authors.name: Duncan Watson-Parris
paper_authors._id: 67282af8fab7cbfa9338980b
paper_authors.name: Leon Bergen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-04T14:24:29.881Z
paper_authors._id: 6303f60ca362e7e8b51d60cd
paper_authors.avatarUrl: /avatars/235be39550fb4fc760aa644df1d5e2d8.svg
paper_authors.fullname: Leon Bergen
paper_authors.type: user
paper_authors.user: lbergen
paper_authors._id: 67282af8fab7cbfa9338980c
paper_authors.name: Taylor Berg-Kirkpatrick
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-04T14:24:35.794Z
paper_authors._id: 66845caa4b898a41b7af775c
paper_authors.avatarUrl: /avatars/e205290cd372b6b464509de279f6642d.svg
paper_authors.fullname: Taylor Berg-Kirkpatrick
paper_authors.type: user
paper_authors.user: tberg12
paper_authors._id: 67282af8fab7cbfa9338980d
paper_authors.name: Rose Yu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-04T02:01:29.621Z
paper_authors._id: 65566b1294d4cb86b0f1d625
paper_authors.avatarUrl: /avatars/995525ad3b282ea32171615030f7bdcf.svg
paper_authors.fullname: Rose Yu
paper_authors.type: user
paper_authors.user: yuqirose
paper_title: Adapting While Learning: Grounding LLMs for Scientific Problems with
    Intelligent Tool Usage Adaptation
paper_summary: Large Language Models (LLMs) demonstrate promising capabilities in solving
  simple scientific problems but often produce hallucinations for complex ones.
  While integrating LLMs with tools can increase reliability, this approach
  typically results in over-reliance on tools, diminishing the model's ability to
  solve simple problems through basic reasoning. In contrast, human experts first
  assess problem complexity using domain knowledge before choosing an appropriate
  solution approach. Inspired by this human problem-solving process, we propose a
  novel two-component fine-tuning method. In the first component World Knowledge
  Distillation (WKD), LLMs learn directly from solutions generated using tool's
  information to internalize domain knowledge. In the second component Tool Usage
  Adaptation (TUA), we partition problems into easy and hard categories based on
  the model's direct answering accuracy. While maintaining the same alignment
  target for easy problems as in WKD, we train the model to intelligently switch
  to tool usage for more challenging problems. We validate our method on six
  scientific benchmark datasets, spanning mathematics, climate science and
  epidemiology. On average, our models demonstrate a 28.18% improvement in answer
  accuracy and a 13.89% increase in tool usage precision across all datasets,
  surpassing state-of-the-art models including GPT-4o and Claude-3.5.
paper_ai_keywords: Large Language Models (LLMs) hallucinations World Knowledge Distillation (WKD) Tool Usage Adaptation (TUA) domain knowledge answer accuracy tool usage precision scientific benchmark datasets mathematics climate science epidemiology GPT-4 Claude-3.5