Democratizing Reasoning Ability: Tailored Learning from Large Language
  Model
Large language models (LLMs) exhibit impressive emergent abilities in natural
language processing, but their democratization is hindered due to huge
computation requirements and closed-source nature. Recent research on advancing
open-source smaller LMs by distilling knowledge from black-box LLMs has
obtained promising results in the instruction-following ability. However, the
reasoning ability which is more challenging to foster, is relatively rarely
explored. In this paper, we propose a tailored learning approach to distill
such reasoning ability to smaller LMs to facilitate the democratization of the
exclusive reasoning ability. In contrast to merely employing LLM as a data
annotator, we exploit the potential of LLM as a reasoning teacher by building
an interactive multi-round learning paradigm. This paradigm enables the student
to expose its deficiencies to the black-box teacher who then can provide
customized training data in return. Further, to exploit the reasoning potential
of the smaller LM, we propose self-reflection learning to motivate the student
to learn from self-made mistakes. The learning from self-reflection and LLM are
all tailored to the student's learning status, thanks to the seamless
integration with the multi-round learning paradigm. Comprehensive experiments
and analysis on mathematical and commonsense reasoning tasks demonstrate the
effectiveness of our method. The code will be available at
https://github.com/Raibows/Learn-to-Reason.
6535ed13a920f38780bf6d09
Zhaoyang Wang
6535ed13a920f38780bf6d0a
Shaohan Huang
6535ed13a920f38780bf6d0b
Yuxuan Liu
6535ed13a920f38780bf6d0c
Jiahai Wang
6535ed13a920f38780bf6d0d
Minghui Song
6535ed13a920f38780bf6d0e
Zihan Zhang
6535ed13a920f38780bf6d0f
Haizhen Huang
admin_assigned
2023-10-23T08:53:26.119Z
6302e8e57e137e3d6b3f5a06
/avatars/aa82fe4fb1212dc120ecccd406568c5d.svg
Huang
user
Haizhen
6535ed13a920f38780bf6d10
Furu Wei
admin_assigned
2023-10-23T08:52:46.443Z
6368c512fbfe97c16a40baba
/avatars/1c23bc7c0b6d9225699ce27647623d7a.svg
Furu Wei
user
thegenerality
6535ed13a920f38780bf6d11
Weiwei Deng
admin_assigned
2023-10-23T08:53:05.184Z
642da49dc1adada5b345c72b
/avatars/5fa8d345a7248246454444999756105b.svg
deng
user
weiweideng
6535ed13a920f38780bf6d12
Feng Sun
admin_assigned
2023-10-23T08:53:12.250Z
64b88a7449bde5d9481e4755
/avatars/45a963517ec2c54c81ee80449d655014.svg
feng sun
user
forserty
6535ed13a920f38780bf6d13
Qi Zhang
Democratizing Reasoning Ability: Tailored Learning from Large Language
  Model
Large language models (LLMs) exhibit impressive emergent abilities in natural
language processing, but their democratization is hindered due to huge
computation requirements and closed-source nature. Recent research on advancing
open-source smaller LMs by distilling knowledge from black-box LLMs has
obtained promising results in the instruction-following ability. However, the
reasoning ability which is more challenging to foster, is relatively rarely
explored. In this paper, we propose a tailored learning approach to distill
such reasoning ability to smaller LMs to facilitate the democratization of the
exclusive reasoning ability. In contrast to merely employing LLM as a data
annotator, we exploit the potential of LLM as a reasoning teacher by building
an interactive multi-round learning paradigm. This paradigm enables the student
to expose its deficiencies to the black-box teacher who then can provide
customized training data in return. Further, to exploit the reasoning potential
of the smaller LM, we propose self-reflection learning to motivate the student
to learn from self-made mistakes. The learning from self-reflection and LLM are
all tailored to the student's learning status, thanks to the seamless
integration with the multi-round learning paradigm. Comprehensive experiments
and analysis on mathematical and commonsense reasoning tasks demonstrate the
effectiveness of our method. The code will be available at
https://github.com/Raibows/Learn-to-Reason.
A tailored learning approach uses interactive multi-round distillation from large language models to enhance reasoning abilities in smaller LMs, fostering democratization through self-reflection and customized training.
large language models LLMS knowledge distillation open-source models instruction-following reasoning ability interactive multi-round learning reasoning teacher self-reflection learning mathematical reasoning commonsense reasoning tasks