title: LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
summary: Pretrained large language models (LLMs) are currently state-of-the-art for
  solving the vast majority of natural language processing tasks. While many
  real-world applications still require fine-tuning to reach satisfactory levels
  of performance, many of them are in the low-data regime, making fine-tuning
  challenging. To address this, we propose LLM2LLM, a targeted and iterative data
  augmentation strategy that uses a teacher LLM to enhance a small seed dataset
  by augmenting additional data that can be used for fine-tuning on a specific
  task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data,
  (2) evaluates and extracts data points that the model gets wrong, and (3) uses
  a teacher LLM to generate synthetic data based on these incorrect data points,
  which are then added back into the training data. This approach amplifies the
  signal from incorrectly predicted data points by the LLM during training and
  reintegrates them into the dataset to focus on more challenging examples for
  the LLM. Our results show that LLM2LLM significantly enhances the performance
  of LLMs in the low-data regime, outperforming both traditional fine-tuning and
  other data augmentation baselines. LLM2LLM reduces the dependence on
  labor-intensive data curation and paves the way for more scalable and
  performant LLM solutions, allowing us to tackle data-constrained domains and
  tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on
  CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular
  fine-tuning in the low-data regime using a LLaMA2-7B student model.
paper_authors._id: 6600fcca4b3187d931c818cd
paper_authors.name: Nicholas Lee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-25T09:50:14.495Z
paper_authors._id: 656a4547996819a82836852d
paper_authors.avatarUrl: /avatars/a5b522c18fced97140a156ff3e88d2e8.svg
paper_authors.fullname: Nicholas Lee
paper_authors.type: user
paper_authors.user: nicholaslee
paper_authors._id: 6600fcca4b3187d931c818ce
paper_authors.name: Thanakul Wattanawong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-25T09:50:29.744Z
paper_authors._id: 6427e76f4fe87caede7e85df
paper_authors.avatarUrl: /avatars/be55e57ae14ef817e4393bbfd2ff14d4.svg
paper_authors.fullname: Wattanawong
paper_authors.type: user
paper_authors.user: Thanakul
paper_authors._id: 6600fcca4b3187d931c818cf
paper_authors.name: Sehoon Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-26T07:54:28.631Z
paper_authors._id: 6022be42aa1483851c6685eb
paper_authors.avatarUrl: /avatars/42975163f9903dfc6ec48b596ca219c5.svg
paper_authors.fullname: Sehoon Kim
paper_authors.type: user
paper_authors.user: kssteven
paper_authors._id: 6600fcca4b3187d931c818d0
paper_authors.name: Karttikeya Mangalam
paper_authors._id: 6600fcca4b3187d931c818d1
paper_authors.name: Sheng Shen
paper_authors._id: 6600fcca4b3187d931c818d2
paper_authors.name: Gopala Anumanchipali
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-25T10:01:05.102Z
paper_authors._id: 60523e4aa7226b25aaeea2b8
paper_authors.avatarUrl: /avatars/316ca348da91ebced86991f36150c959.svg
paper_authors.fullname: Gopala Anumanchipalli
paper_authors.type: user
paper_authors.user: gopalakr
paper_authors._id: 6600fcca4b3187d931c818d3
paper_authors.name: Michael W. Mahoney
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-25T10:01:20.313Z
paper_authors._id: 64e8b009bddc5b1072c428a5
paper_authors.avatarUrl: /avatars/350bac7b36ba1db1ac024a0812aa27c4.svg
paper_authors.fullname: MICHAEL MAHONEY
paper_authors.type: user
paper_authors.user: Mahoney2022
paper_authors._id: 6600fcca4b3187d931c818d4
paper_authors.name: Kurt Keutzer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-25T10:01:29.251Z
paper_authors._id: 6251bf4b183aa4266924ad91
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678041834400-6251bf4b183aa4266924ad91.jpeg
paper_authors.fullname: Kurt Keutzer
paper_authors.type: user
paper_authors.user: kurtkeutzer
paper_authors._id: 6600fcca4b3187d931c818d5
paper_authors.name: Amir Gholami
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-25T10:01:38.368Z
paper_authors._id: 64e456d0384b85306bf5510e
paper_authors.avatarUrl: /avatars/6c838f810c5f655d6b0b0b40211cfad7.svg
paper_authors.fullname: Amir Gholami
paper_authors.type: user
paper_authors.user: amirgh
paper_title: LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
paper_summary: Pretrained large language models (LLMs) are currently state-of-the-art for
  solving the vast majority of natural language processing tasks. While many
  real-world applications still require fine-tuning to reach satisfactory levels
  of performance, many of them are in the low-data regime, making fine-tuning
  challenging. To address this, we propose LLM2LLM, a targeted and iterative data
  augmentation strategy that uses a teacher LLM to enhance a small seed dataset
  by augmenting additional data that can be used for fine-tuning on a specific
  task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data,
  (2) evaluates and extracts data points that the model gets wrong, and (3) uses
  a teacher LLM to generate synthetic data based on these incorrect data points,
  which are then added back into the training data. This approach amplifies the
  signal from incorrectly predicted data points by the LLM during training and
  reintegrates them into the dataset to focus on more challenging examples for
  the LLM. Our results show that LLM2LLM significantly enhances the performance
  of LLMs in the low-data regime, outperforming both traditional fine-tuning and
  other data augmentation baselines. LLM2LLM reduces the dependence on
  labor-intensive data curation and paves the way for more scalable and
  performant LLM solutions, allowing us to tackle data-constrained domains and
  tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on
  CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular
  fine-tuning in the low-data regime using a LLaMA2-7B student model.
paper_ai_summary: LLM2LLM enhances LLM performance in low-data scenarios by iteratively using a teacher model to augment the training dataset with synthetic data based on the student's prediction errors.
paper_ai_keywords: pretrained large language models LLMs fine-tuning low-data regime data augmentation seed dataset baseline student LLM teacher LLM synthetic data evaluation incorrect data points GSM8K CaseHOLD SNIPS TREC SST-2 LLaMA2-7B