Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
This paper introduces rStar, a self-play mutual reasoning approach that
significantly improves reasoning capabilities of small language models (SLMs)
without fine-tuning or superior models. rStar decouples reasoning into a
self-play mutual generation-discrimination process. First, a target SLM
augments the Monte Carlo Tree Search (MCTS) with a rich set of human-like
reasoning actions to construct higher quality reasoning trajectories. Next,
another SLM, with capabilities similar to the target SLM, acts as a
discriminator to verify each trajectory generated by the target SLM. The
mutually agreed reasoning trajectories are considered mutual consistent, thus
are more likely to be correct. Extensive experiments across five SLMs
demonstrate rStar can effectively solve diverse reasoning problems, including
GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA. Remarkably, rStar boosts GSM8K
accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for
Mistral-7B, from 74.53% to 91.13% for LLaMA3-8B-Instruct. Code will be
available at https://github.com/zhentingqi/rStar.
66bad4c6108762ae5cad4135
Zhenting Qi
claimed_verified
2024-08-13T07:38:00.467Z
650390d0c549d0b61839ca13
https://cdn-avatars.huggingface.co/v1/production/uploads/650390d0c549d0b61839ca13/2bFvv7csIA2xVNbVR9RRa.png
Zhenting Qi
user
zhenting
66bad4c6108762ae5cad4136
Mingyuan Ma
claimed_verified
2024-08-14T08:59:16.481Z
64fd5b45c367f7b1ca100818
/avatars/24524454732ddc350e839a9e0589de3d.svg
Mingyuan Ma
user
ThunderBeee
66bad4c6108762ae5cad4137
Jiahang Xu
claimed_verified
2024-08-13T07:38:08.473Z
62abdf657b037eafffc48808
https://cdn-avatars.huggingface.co/v1/production/uploads/1655430982462-noauth.jpeg
Jiahang Xu
user
Jiahang
66bad4c6108762ae5cad4138
Li Lyna Zhang
admin_assigned
2024-08-13T08:54:32.083Z
62b0009c72043b05d29492b2
https://cdn-avatars.huggingface.co/v1/production/uploads/62b0009c72043b05d29492b2/NqRkX2YLhlfOLvYysa7dD.png
Li Lyna Zhang
user
lynazhang
66bad4c6108762ae5cad4139
Fan Yang
66bad4c6108762ae5cad413a
Mao Yang
Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
This paper introduces rStar, a self-play mutual reasoning approach that
significantly improves reasoning capabilities of small language models (SLMs)
without fine-tuning or superior models. rStar decouples reasoning into a
self-play mutual generation-discrimination process. First, a target SLM
augments the Monte Carlo Tree Search (MCTS) with a rich set of human-like
reasoning actions to construct higher quality reasoning trajectories. Next,
another SLM, with capabilities similar to the target SLM, acts as a
discriminator to verify each trajectory generated by the target SLM. The
mutually agreed reasoning trajectories are considered mutual consistent, thus
are more likely to be correct. Extensive experiments across five SLMs
demonstrate rStar can effectively solve diverse reasoning problems, including
GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA. Remarkably, rStar boosts GSM8K
accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for
Mistral-7B, from 74.53% to 91.13% for LLaMA3-8B-Instruct. Code will be
available at https://github.com/zhentingqi/rStar.
rStar enhances small language models' reasoning capabilities through a self-play mutual generation-discrimination process without fine-tuning, improving accuracy across various reasoning tasks.
self-play mutual reasoning small language models Monte Carlo Tree Search reasoning actions reasoning trajectories discrimination mutual consistency