title: Mentor-KD: Making Small Language Models Better Multi-step Reasoners
summary: Large Language Models (LLMs) have displayed remarkable performances across
  various complex tasks by leveraging Chain-of-Thought (CoT) prompting. Recently,
  studies have proposed a Knowledge Distillation (KD) approach, reasoning
  distillation, which transfers such reasoning ability of LLMs through
  fine-tuning language models of multi-step rationales generated by LLM teachers.
  However, they have inadequately considered two challenges regarding
  insufficient distillation sets from the LLM teacher model, in terms of 1) data
  quality and 2) soft label provision. In this paper, we propose Mentor-KD, which
  effectively distills the multi-step reasoning capability of LLMs to smaller LMs
  while addressing the aforementioned challenges. Specifically, we exploit a
  mentor, intermediate-sized task-specific fine-tuned model, to augment
  additional CoT annotations and provide soft labels for the student model during
  reasoning distillation. We conduct extensive experiments and confirm
  Mentor-KD's effectiveness across various models and complex reasoning tasks.
paper_authors._id: 670cefea1699355b61389dc4
paper_authors.name: Hojae Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T10:47:35.460Z
paper_authors._id: 63609fd514657fb8cff9a8b2
paper_authors.avatarUrl: /avatars/421efa9911db82c4c0d6386f0e8c43c7.svg
paper_authors.fullname: hojae
paper_authors.type: user
paper_authors.user: nokomon
paper_authors._id: 670cefea1699355b61389dc5
paper_authors.name: Junho Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T11:38:32.861Z
paper_authors._id: 669891e2bd704c8fb2de6b6d
paper_authors.avatarUrl: /avatars/db492bcb1d41913c271d7fb8cd587494.svg
paper_authors.fullname: Junho Kim
paper_authors.type: user
paper_authors.user: monocrat23
paper_authors._id: 670cefea1699355b61389dc6
paper_authors.name: SangKeun Lee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T11:12:58.738Z
paper_authors._id: 646ec6b5cbb7bb996513c514
paper_authors.avatarUrl: /avatars/24614870dddab1da84da1ec8a5cc8f01.svg
paper_authors.fullname: Sangkeun LEE
paper_authors.type: user
paper_authors.user: SKyii
paper_title: Mentor-KD: Making Small Language Models Better Multi-step Reasoners
paper_summary: Large Language Models (LLMs) have displayed remarkable performances across
  various complex tasks by leveraging Chain-of-Thought (CoT) prompting. Recently,
  studies have proposed a Knowledge Distillation (KD) approach, reasoning
  distillation, which transfers such reasoning ability of LLMs through
  fine-tuning language models of multi-step rationales generated by LLM teachers.
  However, they have inadequately considered two challenges regarding
  insufficient distillation sets from the LLM teacher model, in terms of 1) data
  quality and 2) soft label provision. In this paper, we propose Mentor-KD, which
  effectively distills the multi-step reasoning capability of LLMs to smaller LMs
  while addressing the aforementioned challenges. Specifically, we exploit a
  mentor, intermediate-sized task-specific fine-tuned model, to augment
  additional CoT annotations and provide soft labels for the student model during
  reasoning distillation. We conduct extensive experiments and confirm
  Mentor-KD's effectiveness across various models and complex reasoning tasks.
paper_ai_keywords: Chain-of-Thought (CoT) Knowledge Distillation (KD) Multi-step rationales Fine-tuning Intermediate-sized task-specific fine-tuned model Soft label provision Mentor-KD CoT Annotations