title: Montessori-Instruct: Generate Influential Training Data Tailored for
    Student Learning
summary: Synthetic data has been widely used to train large language models, but their
  generative nature inevitably introduces noisy, non-informative, and misleading
  learning signals. In this paper, we propose Montessori-Instruct, a novel data
  synthesis framework that tailors the data synthesis ability of the teacher
  language model toward the student language model's learning process.
  Specifically, we utilize local data influence of synthetic training data points
  on students to characterize students' learning preferences. Then, we train the
  teacher model with Direct Preference Optimization (DPO) to generate synthetic
  data tailored toward student learning preferences. Experiments with
  Llama3-8B-Instruct (teacher) and Llama3-8B (student) on Alpaca Eval and
  MT-Bench demonstrate that Montessori-Instruct significantly outperforms
  standard synthesis methods by 18.35\% and 46.24\% relatively. Our method also
  beats data synthesized by a stronger teacher model, GPT-4o. Further analysis
  confirms the benefits of teacher's learning to generate more influential
  training data in the student's improved learning, the advantages of local data
  influence in accurately measuring student preferences, and the robustness of
  Montessori-Instruct across different student models. Our code and data are
  open-sourced at https://github.com/cxcscmu/Montessori-Instruct.
paper_authors._id: 671664c4f262be795484c53c
paper_authors.name: Xiaochuan Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T07:23:33.008Z
paper_authors._id: 64b103cf372d434077206750
paper_authors.avatarUrl: /avatars/ba0eb4fc712a8b9b93ceb30d11859ec2.svg
paper_authors.fullname: Xiaochuan Li
paper_authors.type: user
paper_authors.user: lixiaochuan2020
paper_authors._id: 671664c4f262be795484c53d
paper_authors.name: Zichun Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-21T16:15:26.547Z
paper_authors._id: 649a943f575e60d3a87cfcdf
paper_authors.avatarUrl: /avatars/879f79628a30dddeab6e36e7d82bdbaa.svg
paper_authors.fullname:  Zichun Yu
paper_authors.type: user
paper_authors.user: yuzc19
paper_authors._id: 671664c4f262be795484c53e
paper_authors.name: Chenyan Xiong
paper_title: Montessori-Instruct: Generate Influential Training Data Tailored for
    Student Learning
paper_summary: Synthetic data has been widely used to train large language models, but their
  generative nature inevitably introduces noisy, non-informative, and misleading
  learning signals. In this paper, we propose Montessori-Instruct, a novel data
  synthesis framework that tailors the data synthesis ability of the teacher
  language model toward the student language model's learning process.
  Specifically, we utilize local data influence of synthetic training data points
  on students to characterize students' learning preferences. Then, we train the
  teacher model with Direct Preference Optimization (DPO) to generate synthetic
  data tailored toward student learning preferences. Experiments with
  Llama3-8B-Instruct (teacher) and Llama3-8B (student) on Alpaca Eval and
  MT-Bench demonstrate that Montessori-Instruct significantly outperforms
  standard synthesis methods by 18.35\% and 46.24\% relatively. Our method also
  beats data synthesized by a stronger teacher model, GPT-4o. Further analysis
  confirms the benefits of teacher's learning to generate more influential
  training data in the student's improved learning, the advantages of local data
  influence in accurately measuring student preferences, and the robustness of
  Montessori-Instruct across different student models. Our code and data are
  open-sourced at https://github.com/cxcscmu/Montessori-Instruct.
paper_ai_summary: Montessori-Instruct enhances language model training by optimizing synthetic data generation to align with student learning preferences, outperforming standard methods and a more powerful teacher model.
paper_ai_keywords: local data influence Direct Preference Optimization (DPO) Alpaca Eval MT-Bench Llama3-8B-Instruct Llama3-8B GPT-4o