Aligning Teacher with Student Preferences for Tailored Training Data
  Generation
Large Language Models (LLMs) have shown significant promise as copilots in
various tasks. Local deployment of LLMs on edge devices is necessary when
handling privacy-sensitive data or latency-sensitive tasks. The computational
constraints of such devices make direct deployment of powerful large-scale LLMs
impractical, necessitating the Knowledge Distillation from large-scale models
to lightweight models. Lots of work has been done to elicit diversity and
quality training examples from LLMs, but little attention has been paid to
aligning teacher instructional content based on student preferences, akin to
"responsive teaching" in pedagogy. Thus, we propose ARTE, dubbed Aligning
TeacheR with StudenT PreferencEs, a framework that aligns the teacher model
with student preferences to generate tailored training examples for Knowledge
Distillation. Specifically, we elicit draft questions and rationales from the
teacher model, then collect student preferences on these questions and
rationales using students' performance with in-context learning as a proxy, and
finally align the teacher model with student preferences. In the end, we repeat
the first step with the aligned teacher model to elicit tailored training
examples for the student model on the target task. Extensive experiments on
academic benchmarks demonstrate the superiority of ARTE over existing
instruction-tuning datasets distilled from powerful LLMs. Moreover, we
thoroughly investigate the generalization of ARTE, including the generalization
of fine-tuned student models in reasoning ability and the generalization of
aligned teacher models to generate tailored training data across tasks and
students. In summary, our contributions lie in proposing a novel framework for
tailored training example generation, demonstrating its efficacy in
experiments, and investigating the generalization of both student & aligned
teacher models in ARTE.
667e1e2ad29e971c593931a0
Yantao Liu
667e1e2ad29e971c593931a1
Zhao Zhang
667e1e2ad29e971c593931a2
Zijun Yao
claimed_verified
2024-06-28T08:02:57.469Z
648c4b46e549be47af1aafcd
https://cdn-avatars.huggingface.co/v1/production/uploads/648c4b46e549be47af1aafcd/YgOHbmUM2EDM-lb7GdqXz.jpeg
Zijun
user
TranSirius
667e1e2ad29e971c593931a3
Shulin Cao
admin_assigned
2024-06-28T12:11:21.575Z
62c924a6334a6ee11c2e8dfa
/avatars/3dbc37af162b94d68cb83665ac4528c3.svg
ShulinCao
user
caoshulin
667e1e2ad29e971c593931a4
Lei Hou
667e1e2ad29e971c593931a5
Juanzi Li
admin_assigned
2024-06-28T12:11:10.760Z
65df8cbc2705d9672f55d1aa
/avatars/63e46f15bb76bd9d4508fd0f54f39829.svg
Juanzi Li
user
juanli
Aligning Teacher with Student Preferences for Tailored Training Data
  Generation
Large Language Models (LLMs) have shown significant promise as copilots in
various tasks. Local deployment of LLMs on edge devices is necessary when
handling privacy-sensitive data or latency-sensitive tasks. The computational
constraints of such devices make direct deployment of powerful large-scale LLMs
impractical, necessitating the Knowledge Distillation from large-scale models
to lightweight models. Lots of work has been done to elicit diversity and
quality training examples from LLMs, but little attention has been paid to
aligning teacher instructional content based on student preferences, akin to
"responsive teaching" in pedagogy. Thus, we propose ARTE, dubbed Aligning
TeacheR with StudenT PreferencEs, a framework that aligns the teacher model
with student preferences to generate tailored training examples for Knowledge
Distillation. Specifically, we elicit draft questions and rationales from the
teacher model, then collect student preferences on these questions and
rationales using students' performance with in-context learning as a proxy, and
finally align the teacher model with student preferences. In the end, we repeat
the first step with the aligned teacher model to elicit tailored training
examples for the student model on the target task. Extensive experiments on
academic benchmarks demonstrate the superiority of ARTE over existing
instruction-tuning datasets distilled from powerful LLMs. Moreover, we
thoroughly investigate the generalization of ARTE, including the generalization
of fine-tuned student models in reasoning ability and the generalization of
aligned teacher models to generate tailored training data across tasks and
students. In summary, our contributions lie in proposing a novel framework for
tailored training example generation, demonstrating its efficacy in
experiments, and investigating the generalization of both student & aligned
teacher models in ARTE.
ARTE, a framework that aligns teacher models with student preferences, enhances knowledge distillation for edge deployment of LLMs, demonstrating superior performance and generalization compared to existing methods.
LLMs Knowledge Distillation responsive teaching ARTE draft questions rationales in-context learning student preferences tailored training examples fine-tuned student models reasoning ability aligned teacher models