title: START: Self-taught Reasoner with Tools
summary: Large reasoning models (LRMs) like OpenAI-o1 and DeepSeek-R1 have
  demonstrated remarkable capabilities in complex reasoning tasks through the
  utilization of long Chain-of-thought (CoT). However, these models often suffer
  from hallucinations and inefficiencies due to their reliance solely on internal
  reasoning processes. In this paper, we introduce START (Self-Taught Reasoner
  with Tools), a novel tool-integrated long CoT reasoning LLM that significantly
  enhances reasoning capabilities by leveraging external tools. Through code
  execution, START is capable of performing complex computations, self-checking,
  exploring diverse methods, and self-debugging, thereby addressing the
  limitations of LRMs. The core innovation of START lies in its self-learning
  framework, which comprises two key techniques: 1) Hint-infer: We demonstrate
  that inserting artificially designed hints (e.g., ``Wait, maybe using Python
  here is a good idea.'') during the inference process of a LRM effectively
  stimulates its ability to utilize external tools without the need for any
  demonstration data. Hint-infer can also serve as a simple and effective
  sequential test-time scaling method; 2) Hint Rejection Sampling Fine-Tuning
  (Hint-RFT): Hint-RFT combines Hint-infer and RFT by scoring, filtering, and
  modifying the reasoning trajectories with tool invocation generated by a LRM
  via Hint-infer, followed by fine-tuning the LRM. Through this framework, we
  have fine-tuned the QwQ-32B model to achieve START. On PhD-level science QA
  (GPQA), competition-level math benchmarks (AMC23, AIME24, AIME25), and the
  competition-level code benchmark (LiveCodeBench), START achieves accuracy rates
  of 63.6%, 95.0%, 66.7%, 47.1%, and 47.3%, respectively. It significantly
  outperforms the base QwQ-32B and achieves performance comparable to the
  state-of-the-art open-weight model R1-Distill-Qwen-32B and the proprietary
  model o1-Preview.
paper_authors._id: 67ca670d3e81e3344dc4c2d9
paper_authors.name: Chengpeng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:12:37.350Z
paper_authors._id: 65294b334d7cf551ac50d6a6
paper_authors.avatarUrl: /avatars/75d21e20b711b871616ef3850bb900b7.svg
paper_authors.fullname: ChengpengLi
paper_authors.type: user
paper_authors.user: ChengpengLi
paper_authors._id: 67ca670d3e81e3344dc4c2da
paper_authors.name: Mingfeng Xue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:12:28.354Z
paper_authors._id: 5f8946925d083370c711f296
paper_authors.avatarUrl: /avatars/14246aae3b1f8b7ad050f8ff2c8b260e.svg
paper_authors.fullname: Mingfeng Xue
paper_authors.type: user
paper_authors.user: mingfengxue
paper_authors._id: 67ca670d3e81e3344dc4c2db
paper_authors.name: Zhenru Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:12:48.194Z
paper_authors._id: 64704e973601bb7b06643e98
paper_authors.avatarUrl: /avatars/52e51f4d1be6769e4397b8be2799cf32.svg
paper_authors.fullname: Zhenru Zhang
paper_authors.type: user
paper_authors.user: Zhenru
paper_authors._id: 67ca670d3e81e3344dc4c2dc
paper_authors.name: Jiaxi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:12:57.082Z
paper_authors._id: 646df403ad20c6fa4f30b7ec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646df403ad20c6fa4f30b7ec/Q64-XMghOcBoo3itZDGYA.jpeg
paper_authors.fullname: Jiaxi Yang
paper_authors.type: user
paper_authors.user: jx-yang
paper_authors._id: 67ca670d3e81e3344dc4c2dd
paper_authors.name: Beichen Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:13:11.641Z
paper_authors._id: 64b93578ee257c3a4cfceed1
paper_authors.avatarUrl: /avatars/e6188562254f75a09b4048b800860016.svg
paper_authors.fullname: Beichen Zhang
paper_authors.type: user
paper_authors.user: BeichenZhang
paper_authors._id: 67ca670d3e81e3344dc4c2de
paper_authors.name: Xiang Wang
paper_authors._id: 67ca670d3e81e3344dc4c2df
paper_authors.name: Bowen Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:13:30.530Z
paper_authors._id: 6583ab7983a9e1460c67d876
paper_authors.avatarUrl: /avatars/74400bc448c3f07e23a4cd53d68a6af7.svg
paper_authors.fullname: bowen
paper_authors.type: user
paper_authors.user: bowenYu
paper_authors._id: 67ca670d3e81e3344dc4c2e0
paper_authors.name: Binyuan Hui
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:13:37.341Z
paper_authors._id: 61e4c4ca1ab24785ac11ba69
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61e4c4ca1ab24785ac11ba69/1Q1zhhyGSJ9RJG9MzwxVv.jpeg
paper_authors.fullname: Binyuan Hui
paper_authors.type: user
paper_authors.user: huybery
paper_authors._id: 67ca670d3e81e3344dc4c2e1
paper_authors.name: Junyang Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:13:44.084Z
paper_authors._id: 620760a26e3b7210c2ff1943
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/VC-rKqimF6yxGESNVlPoR.jpeg
paper_authors.fullname: Junyang Lin
paper_authors.type: user
paper_authors.user: JustinLin610
paper_authors._id: 67ca670d3e81e3344dc4c2e2
paper_authors.name: Dayiheng Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-07T09:13:52.711Z
paper_authors._id: 6434d4989bd5a84b5dd0b0f5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6434d4989bd5a84b5dd0b0f5/0Elf9qbfG9Hkgypm9pTGm.jpeg
paper_authors.fullname: Dayiheng Liu
paper_authors.type: user
paper_authors.user: Losin94
paper_title: START: Self-taught Reasoner with Tools
paper_summary: Large reasoning models (LRMs) like OpenAI-o1 and DeepSeek-R1 have
  demonstrated remarkable capabilities in complex reasoning tasks through the
  utilization of long Chain-of-thought (CoT). However, these models often suffer
  from hallucinations and inefficiencies due to their reliance solely on internal
  reasoning processes. In this paper, we introduce START (Self-Taught Reasoner
  with Tools), a novel tool-integrated long CoT reasoning LLM that significantly
  enhances reasoning capabilities by leveraging external tools. Through code
  execution, START is capable of performing complex computations, self-checking,
  exploring diverse methods, and self-debugging, thereby addressing the
  limitations of LRMs. The core innovation of START lies in its self-learning
  framework, which comprises two key techniques: 1) Hint-infer: We demonstrate
  that inserting artificially designed hints (e.g., ``Wait, maybe using Python
  here is a good idea.'') during the inference process of a LRM effectively
  stimulates its ability to utilize external tools without the need for any
  demonstration data. Hint-infer can also serve as a simple and effective
  sequential test-time scaling method; 2) Hint Rejection Sampling Fine-Tuning
  (Hint-RFT): Hint-RFT combines Hint-infer and RFT by scoring, filtering, and
  modifying the reasoning trajectories with tool invocation generated by a LRM
  via Hint-infer, followed by fine-tuning the LRM. Through this framework, we
  have fine-tuned the QwQ-32B model to achieve START. On PhD-level science QA
  (GPQA), competition-level math benchmarks (AMC23, AIME24, AIME25), and the
  competition-level code benchmark (LiveCodeBench), START achieves accuracy rates
  of 63.6%, 95.0%, 66.7%, 47.1%, and 47.3%, respectively. It significantly
  outperforms the base QwQ-32B and achieves performance comparable to the
  state-of-the-art open-weight model R1-Distill-Qwen-32B and the proprietary
  model o1-Preview.
paper_ai_keywords: Chain-of-thought (CoT) hallucinations inefficiencies long CoT reasoning LLM tool-integrated code execution self-checking self-debugging self-learning framework Hint-infer Hint Rejection Sampling Fine-Tuning (Hint-RFT) RFT reasoning trajectories tool invocation PhD-level science QA (GPQA) competition-level math benchmarks (AMC23, AIME24, AIME25) competition-level code benchmark (LiveCodeBench)