title: Democratizing Reasoning Ability: Tailored Learning from Large Language
    Model
summary: Large language models (LLMs) exhibit impressive emergent abilities in natural
  language processing, but their democratization is hindered due to huge
  computation requirements and closed-source nature. Recent research on advancing
  open-source smaller LMs by distilling knowledge from black-box LLMs has
  obtained promising results in the instruction-following ability. However, the
  reasoning ability which is more challenging to foster, is relatively rarely
  explored. In this paper, we propose a tailored learning approach to distill
  such reasoning ability to smaller LMs to facilitate the democratization of the
  exclusive reasoning ability. In contrast to merely employing LLM as a data
  annotator, we exploit the potential of LLM as a reasoning teacher by building
  an interactive multi-round learning paradigm. This paradigm enables the student
  to expose its deficiencies to the black-box teacher who then can provide
  customized training data in return. Further, to exploit the reasoning potential
  of the smaller LM, we propose self-reflection learning to motivate the student
  to learn from self-made mistakes. The learning from self-reflection and LLM are
  all tailored to the student's learning status, thanks to the seamless
  integration with the multi-round learning paradigm. Comprehensive experiments
  and analysis on mathematical and commonsense reasoning tasks demonstrate the
  effectiveness of our method. The code will be available at
  https://github.com/Raibows/Learn-to-Reason.
paper_authors._id: 6535ed13a920f38780bf6d09
paper_authors.name: Zhaoyang Wang
paper_authors._id: 6535ed13a920f38780bf6d0a
paper_authors.name: Shaohan Huang
paper_authors._id: 6535ed13a920f38780bf6d0b
paper_authors.name: Yuxuan Liu
paper_authors._id: 6535ed13a920f38780bf6d0c
paper_authors.name: Jiahai Wang
paper_authors._id: 6535ed13a920f38780bf6d0d
paper_authors.name: Minghui Song
paper_authors._id: 6535ed13a920f38780bf6d0e
paper_authors.name: Zihan Zhang
paper_authors._id: 6535ed13a920f38780bf6d0f
paper_authors.name: Haizhen Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-23T08:53:26.119Z
paper_authors._id: 6302e8e57e137e3d6b3f5a06
paper_authors.avatarUrl: /avatars/aa82fe4fb1212dc120ecccd406568c5d.svg
paper_authors.fullname: Huang
paper_authors.type: user
paper_authors.user: Haizhen
paper_authors._id: 6535ed13a920f38780bf6d10
paper_authors.name: Furu Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-23T08:52:46.443Z
paper_authors._id: 6368c512fbfe97c16a40baba
paper_authors.avatarUrl: /avatars/1c23bc7c0b6d9225699ce27647623d7a.svg
paper_authors.fullname: Furu Wei
paper_authors.type: user
paper_authors.user: thegenerality
paper_authors._id: 6535ed13a920f38780bf6d11
paper_authors.name: Weiwei Deng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-23T08:53:05.184Z
paper_authors._id: 642da49dc1adada5b345c72b
paper_authors.avatarUrl: /avatars/5fa8d345a7248246454444999756105b.svg
paper_authors.fullname: deng
paper_authors.type: user
paper_authors.user: weiweideng
paper_authors._id: 6535ed13a920f38780bf6d12
paper_authors.name: Feng Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-23T08:53:12.250Z
paper_authors._id: 64b88a7449bde5d9481e4755
paper_authors.avatarUrl: /avatars/45a963517ec2c54c81ee80449d655014.svg
paper_authors.fullname: feng sun
paper_authors.type: user
paper_authors.user: forserty
paper_authors._id: 6535ed13a920f38780bf6d13
paper_authors.name: Qi Zhang
paper_title: Democratizing Reasoning Ability: Tailored Learning from Large Language
    Model
paper_summary: Large language models (LLMs) exhibit impressive emergent abilities in natural
  language processing, but their democratization is hindered due to huge
  computation requirements and closed-source nature. Recent research on advancing
  open-source smaller LMs by distilling knowledge from black-box LLMs has
  obtained promising results in the instruction-following ability. However, the
  reasoning ability which is more challenging to foster, is relatively rarely
  explored. In this paper, we propose a tailored learning approach to distill
  such reasoning ability to smaller LMs to facilitate the democratization of the
  exclusive reasoning ability. In contrast to merely employing LLM as a data
  annotator, we exploit the potential of LLM as a reasoning teacher by building
  an interactive multi-round learning paradigm. This paradigm enables the student
  to expose its deficiencies to the black-box teacher who then can provide
  customized training data in return. Further, to exploit the reasoning potential
  of the smaller LM, we propose self-reflection learning to motivate the student
  to learn from self-made mistakes. The learning from self-reflection and LLM are
  all tailored to the student's learning status, thanks to the seamless
  integration with the multi-round learning paradigm. Comprehensive experiments
  and analysis on mathematical and commonsense reasoning tasks demonstrate the
  effectiveness of our method. The code will be available at
  https://github.com/Raibows/Learn-to-Reason.
paper_ai_summary: A tailored learning approach uses interactive multi-round distillation from large language models to enhance reasoning abilities in smaller LMs, fostering democratization through self-reflection and customized training.
paper_ai_keywords: large language models LLMS knowledge distillation open-source models instruction-following reasoning ability interactive multi-round learning reasoning teacher self-reflection learning mathematical reasoning commonsense reasoning tasks