title: Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
summary: This paper introduces rStar, a self-play mutual reasoning approach that
  significantly improves reasoning capabilities of small language models (SLMs)
  without fine-tuning or superior models. rStar decouples reasoning into a
  self-play mutual generation-discrimination process. First, a target SLM
  augments the Monte Carlo Tree Search (MCTS) with a rich set of human-like
  reasoning actions to construct higher quality reasoning trajectories. Next,
  another SLM, with capabilities similar to the target SLM, acts as a
  discriminator to verify each trajectory generated by the target SLM. The
  mutually agreed reasoning trajectories are considered mutual consistent, thus
  are more likely to be correct. Extensive experiments across five SLMs
  demonstrate rStar can effectively solve diverse reasoning problems, including
  GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA. Remarkably, rStar boosts GSM8K
  accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for
  Mistral-7B, from 74.53% to 91.13% for LLaMA3-8B-Instruct. Code will be
  available at https://github.com/zhentingqi/rStar.
paper_authors._id: 66bad4c6108762ae5cad4135
paper_authors.name: Zhenting Qi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-13T07:38:00.467Z
paper_authors._id: 650390d0c549d0b61839ca13
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650390d0c549d0b61839ca13/2bFvv7csIA2xVNbVR9RRa.png
paper_authors.fullname: Zhenting Qi
paper_authors.type: user
paper_authors.user: zhenting
paper_authors._id: 66bad4c6108762ae5cad4136
paper_authors.name: Mingyuan Ma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-14T08:59:16.481Z
paper_authors._id: 64fd5b45c367f7b1ca100818
paper_authors.avatarUrl: /avatars/24524454732ddc350e839a9e0589de3d.svg
paper_authors.fullname: Mingyuan Ma
paper_authors.type: user
paper_authors.user: ThunderBeee
paper_authors._id: 66bad4c6108762ae5cad4137
paper_authors.name: Jiahang Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-13T07:38:08.473Z
paper_authors._id: 62abdf657b037eafffc48808
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1655430982462-noauth.jpeg
paper_authors.fullname: Jiahang Xu
paper_authors.type: user
paper_authors.user: Jiahang
paper_authors._id: 66bad4c6108762ae5cad4138
paper_authors.name: Li Lyna Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T08:54:32.083Z
paper_authors._id: 62b0009c72043b05d29492b2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b0009c72043b05d29492b2/NqRkX2YLhlfOLvYysa7dD.png
paper_authors.fullname: Li Lyna Zhang
paper_authors.type: user
paper_authors.user: lynazhang
paper_authors._id: 66bad4c6108762ae5cad4139
paper_authors.name: Fan Yang
paper_authors._id: 66bad4c6108762ae5cad413a
paper_authors.name: Mao Yang
paper_title: Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
paper_summary: This paper introduces rStar, a self-play mutual reasoning approach that
  significantly improves reasoning capabilities of small language models (SLMs)
  without fine-tuning or superior models. rStar decouples reasoning into a
  self-play mutual generation-discrimination process. First, a target SLM
  augments the Monte Carlo Tree Search (MCTS) with a rich set of human-like
  reasoning actions to construct higher quality reasoning trajectories. Next,
  another SLM, with capabilities similar to the target SLM, acts as a
  discriminator to verify each trajectory generated by the target SLM. The
  mutually agreed reasoning trajectories are considered mutual consistent, thus
  are more likely to be correct. Extensive experiments across five SLMs
  demonstrate rStar can effectively solve diverse reasoning problems, including
  GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA. Remarkably, rStar boosts GSM8K
  accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for
  Mistral-7B, from 74.53% to 91.13% for LLaMA3-8B-Instruct. Code will be
  available at https://github.com/zhentingqi/rStar.
paper_ai_summary: rStar enhances small language models' reasoning capabilities through a self-play mutual generation-discrimination process without fine-tuning, improving accuracy across various reasoning tasks.
paper_ai_keywords: self-play mutual reasoning small language models Monte Carlo Tree Search reasoning actions reasoning trajectories discrimination mutual consistency