Mentor-KD: Making Small Language Models Better Multi-step Reasoners
Large Language Models (LLMs) have displayed remarkable performances across
various complex tasks by leveraging Chain-of-Thought (CoT) prompting. Recently,
studies have proposed a Knowledge Distillation (KD) approach, reasoning
distillation, which transfers such reasoning ability of LLMs through
fine-tuning language models of multi-step rationales generated by LLM teachers.
However, they have inadequately considered two challenges regarding
insufficient distillation sets from the LLM teacher model, in terms of 1) data
quality and 2) soft label provision. In this paper, we propose Mentor-KD, which
effectively distills the multi-step reasoning capability of LLMs to smaller LMs
while addressing the aforementioned challenges. Specifically, we exploit a
mentor, intermediate-sized task-specific fine-tuned model, to augment
additional CoT annotations and provide soft labels for the student model during
reasoning distillation. We conduct extensive experiments and confirm
Mentor-KD's effectiveness across various models and complex reasoning tasks.
670cefea1699355b61389dc4
Hojae Lee
claimed_verified
2024-10-14T10:47:35.460Z
63609fd514657fb8cff9a8b2
/avatars/421efa9911db82c4c0d6386f0e8c43c7.svg
hojae
user
nokomon
670cefea1699355b61389dc5
Junho Kim
claimed_verified
2024-10-14T11:38:32.861Z
669891e2bd704c8fb2de6b6d
/avatars/db492bcb1d41913c271d7fb8cd587494.svg
Junho Kim
user
monocrat23
670cefea1699355b61389dc6
SangKeun Lee
admin_assigned
2024-10-14T11:12:58.738Z
646ec6b5cbb7bb996513c514
/avatars/24614870dddab1da84da1ec8a5cc8f01.svg
Sangkeun LEE
user
SKyii
Mentor-KD: Making Small Language Models Better Multi-step Reasoners
Large Language Models (LLMs) have displayed remarkable performances across
various complex tasks by leveraging Chain-of-Thought (CoT) prompting. Recently,
studies have proposed a Knowledge Distillation (KD) approach, reasoning
distillation, which transfers such reasoning ability of LLMs through
fine-tuning language models of multi-step rationales generated by LLM teachers.
However, they have inadequately considered two challenges regarding
insufficient distillation sets from the LLM teacher model, in terms of 1) data
quality and 2) soft label provision. In this paper, we propose Mentor-KD, which
effectively distills the multi-step reasoning capability of LLMs to smaller LMs
while addressing the aforementioned challenges. Specifically, we exploit a
mentor, intermediate-sized task-specific fine-tuned model, to augment
additional CoT annotations and provide soft labels for the student model during
reasoning distillation. We conduct extensive experiments and confirm
Mentor-KD's effectiveness across various models and complex reasoning tasks.
Mentor-KD enhances the reasoning ability distillation from large to smaller language models by using an intermediate mentor model to provide additional chain-of-thought annotations and soft labels.
Chain-of-Thought Knowledge Distillation reasoning distillation fine-tuning multi-step rationales student model mentor model complex reasoning tasks