title: DELIFT: Data Efficient Language model Instruction Fine Tuning
summary: Fine-tuning large language models (LLMs) is essential for enhancing their
  performance on specific tasks but is often resource-intensive due to redundant
  or uninformative data. To address this inefficiency, we introduce DELIFT (Data
  Efficient Language model Instruction Fine-Tuning), a novel algorithm that
  systematically optimizes data selection across the three key stages of
  fine-tuning: (1) instruction tuning, (2) task-specific fine-tuning (e.g.,
  reasoning, question-answering), and (3) continual fine-tuning (e.g.,
  incorporating new data versions). Unlike existing methods that focus on
  single-stage optimization or rely on computationally intensive gradient
  calculations, DELIFT operates efficiently across all stages. Central to our
  approach is a pairwise utility metric that quantifies how beneficial a data
  sample is for improving the model's responses to other samples, effectively
  measuring the informational value relative to the model's current capabilities.
  By leveraging different submodular functions applied to this metric, DELIFT
  selects diverse and optimal subsets that are useful across all stages of
  fine-tuning. Experiments across various tasks and model scales demonstrate that
  DELIFT can reduce the fine-tuning data size by up to 70% without compromising
  performance, offering significant computational savings and outperforming
  existing methods in both efficiency and efficacy.
paper_authors._id: 67319dedb26b12f7069a16f4
paper_authors.name: Ishika Agarwal
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-02-18T18:06:35.675Z
paper_authors._id: 6391e4e984afa726d66180b9
paper_authors.avatarUrl: /avatars/e437e2820745b522a868b8da27d9a11f.svg
paper_authors.fullname: Ishika Agarwal
paper_authors.type: user
paper_authors.user: ishikaa
paper_authors._id: 67319dedb26b12f7069a16f5
paper_authors.name: Krishna Killamsetty
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-17T08:06:17.827Z
paper_authors._id: 64e4f834a07739bcc4906a3f
paper_authors.avatarUrl: /avatars/19df353130dfdfb826ec0aac28904d5c.svg
paper_authors.fullname: Krishnateja Killamsetty
paper_authors.type: user
paper_authors.user: krishnatejak
paper_authors._id: 67319dedb26b12f7069a16f6
paper_authors.name: Lucian Popa
paper_authors._id: 67319dedb26b12f7069a16f7
paper_authors.name: Marina Danilevksy
paper_title: DELIFT: Data Efficient Language model Instruction Fine Tuning
paper_summary: Fine-tuning large language models (LLMs) is essential for enhancing their
  performance on specific tasks but is often resource-intensive due to redundant
  or uninformative data. To address this inefficiency, we introduce DELIFT (Data
  Efficient Language model Instruction Fine-Tuning), a novel algorithm that
  systematically optimizes data selection across the three key stages of
  fine-tuning: (1) instruction tuning, (2) task-specific fine-tuning (e.g.,
  reasoning, question-answering), and (3) continual fine-tuning (e.g.,
  incorporating new data versions). Unlike existing methods that focus on
  single-stage optimization or rely on computationally intensive gradient
  calculations, DELIFT operates efficiently across all stages. Central to our
  approach is a pairwise utility metric that quantifies how beneficial a data
  sample is for improving the model's responses to other samples, effectively
  measuring the informational value relative to the model's current capabilities.
  By leveraging different submodular functions applied to this metric, DELIFT
  selects diverse and optimal subsets that are useful across all stages of
  fine-tuning. Experiments across various tasks and model scales demonstrate that
  DELIFT can reduce the fine-tuning data size by up to 70% without compromising
  performance, offering significant computational savings and outperforming
  existing methods in both efficiency and efficacy.
paper_ai_summary: DELIFT optimizes data selection during LLM fine-tuning across multiple stages using a pairwise utility metric, reducing data size and computational costs while maintaining high performance.
paper_ai_keywords: DELIFT large language models fine-tuning instruction tuning task-specific fine-tuning continual fine-tuning pairwise utility metric informational value submodular functions data efficiency computational savings