title: Aligning Teacher with Student Preferences for Tailored Training Data
    Generation
summary: Large Language Models (LLMs) have shown significant promise as copilots in
  various tasks. Local deployment of LLMs on edge devices is necessary when
  handling privacy-sensitive data or latency-sensitive tasks. The computational
  constraints of such devices make direct deployment of powerful large-scale LLMs
  impractical, necessitating the Knowledge Distillation from large-scale models
  to lightweight models. Lots of work has been done to elicit diversity and
  quality training examples from LLMs, but little attention has been paid to
  aligning teacher instructional content based on student preferences, akin to
  "responsive teaching" in pedagogy. Thus, we propose ARTE, dubbed Aligning
  TeacheR with StudenT PreferencEs, a framework that aligns the teacher model
  with student preferences to generate tailored training examples for Knowledge
  Distillation. Specifically, we elicit draft questions and rationales from the
  teacher model, then collect student preferences on these questions and
  rationales using students' performance with in-context learning as a proxy, and
  finally align the teacher model with student preferences. In the end, we repeat
  the first step with the aligned teacher model to elicit tailored training
  examples for the student model on the target task. Extensive experiments on
  academic benchmarks demonstrate the superiority of ARTE over existing
  instruction-tuning datasets distilled from powerful LLMs. Moreover, we
  thoroughly investigate the generalization of ARTE, including the generalization
  of fine-tuned student models in reasoning ability and the generalization of
  aligned teacher models to generate tailored training data across tasks and
  students. In summary, our contributions lie in proposing a novel framework for
  tailored training example generation, demonstrating its efficacy in
  experiments, and investigating the generalization of both student & aligned
  teacher models in ARTE.
paper_authors._id: 667e1e2ad29e971c593931a0
paper_authors.name: Yantao Liu
paper_authors._id: 667e1e2ad29e971c593931a1
paper_authors.name: Zhao Zhang
paper_authors._id: 667e1e2ad29e971c593931a2
paper_authors.name: Zijun Yao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-28T08:02:57.469Z
paper_authors._id: 648c4b46e549be47af1aafcd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/648c4b46e549be47af1aafcd/YgOHbmUM2EDM-lb7GdqXz.jpeg
paper_authors.fullname: Zijun
paper_authors.type: user
paper_authors.user: TranSirius
paper_authors._id: 667e1e2ad29e971c593931a3
paper_authors.name: Shulin Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-28T12:11:21.575Z
paper_authors._id: 62c924a6334a6ee11c2e8dfa
paper_authors.avatarUrl: /avatars/3dbc37af162b94d68cb83665ac4528c3.svg
paper_authors.fullname: ShulinCao
paper_authors.type: user
paper_authors.user: caoshulin
paper_authors._id: 667e1e2ad29e971c593931a4
paper_authors.name: Lei Hou
paper_authors._id: 667e1e2ad29e971c593931a5
paper_authors.name: Juanzi Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-28T12:11:10.760Z
paper_authors._id: 65df8cbc2705d9672f55d1aa
paper_authors.avatarUrl: /avatars/63e46f15bb76bd9d4508fd0f54f39829.svg
paper_authors.fullname: Juanzi Li
paper_authors.type: user
paper_authors.user: juanli
paper_title: Aligning Teacher with Student Preferences for Tailored Training Data
    Generation
paper_summary: Large Language Models (LLMs) have shown significant promise as copilots in
  various tasks. Local deployment of LLMs on edge devices is necessary when
  handling privacy-sensitive data or latency-sensitive tasks. The computational
  constraints of such devices make direct deployment of powerful large-scale LLMs
  impractical, necessitating the Knowledge Distillation from large-scale models
  to lightweight models. Lots of work has been done to elicit diversity and
  quality training examples from LLMs, but little attention has been paid to
  aligning teacher instructional content based on student preferences, akin to
  "responsive teaching" in pedagogy. Thus, we propose ARTE, dubbed Aligning
  TeacheR with StudenT PreferencEs, a framework that aligns the teacher model
  with student preferences to generate tailored training examples for Knowledge
  Distillation. Specifically, we elicit draft questions and rationales from the
  teacher model, then collect student preferences on these questions and
  rationales using students' performance with in-context learning as a proxy, and
  finally align the teacher model with student preferences. In the end, we repeat
  the first step with the aligned teacher model to elicit tailored training
  examples for the student model on the target task. Extensive experiments on
  academic benchmarks demonstrate the superiority of ARTE over existing
  instruction-tuning datasets distilled from powerful LLMs. Moreover, we
  thoroughly investigate the generalization of ARTE, including the generalization
  of fine-tuned student models in reasoning ability and the generalization of
  aligned teacher models to generate tailored training data across tasks and
  students. In summary, our contributions lie in proposing a novel framework for
  tailored training example generation, demonstrating its efficacy in
  experiments, and investigating the generalization of both student & aligned
  teacher models in ARTE.
paper_ai_summary: ARTE, a framework that aligns teacher models with student preferences, enhances knowledge distillation for edge deployment of LLMs, demonstrating superior performance and generalization compared to existing methods.
paper_ai_keywords: LLMs Knowledge Distillation responsive teaching ARTE draft questions rationales in-context learning student preferences tailored training examples fine-tuned student models reasoning ability aligned teacher models