title: Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test
    Generation: An Empirical Study
summary: The advent of large language models (LLMs) like GitHub Copilot has
  significantly enhanced programmers' productivity, particularly in code
  generation. However, these models often struggle with real-world tasks without
  fine-tuning. As LLMs grow larger and more performant, fine-tuning for
  specialized tasks becomes increasingly expensive. Parameter-efficient
  fine-tuning (PEFT) methods, which fine-tune only a subset of model parameters,
  offer a promising solution by reducing the computational costs of tuning LLMs
  while maintaining their performance. Existing studies have explored using PEFT
  and LLMs for various code-related tasks and found that the effectiveness of
  PEFT techniques is task-dependent. The application of PEFT techniques in unit
  test generation remains underexplored. The state-of-the-art is limited to using
  LLMs with full fine-tuning to generate unit tests. This paper investigates both
  full fine-tuning and various PEFT methods, including LoRA, (IA)^3, and prompt
  tuning, across different model architectures and sizes. We use well-established
  benchmark datasets to evaluate their effectiveness in unit test generation. Our
  findings show that PEFT methods can deliver performance comparable to full
  fine-tuning for unit test generation, making specialized fine-tuning more
  accessible and cost-effective. Notably, prompt tuning is the most effective in
  terms of cost and resource utilization, while LoRA approaches the effectiveness
  of full fine-tuning in several cases.
paper_authors._id: 673171234edd034cb4bdaf73
paper_authors.name: André Storhaug
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-11-11T04:17:16.529Z
paper_authors._id: 621997b88e4346d1e0f23bc9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/621997b88e4346d1e0f23bc9/TJmde19Ef6t4MB3PH-zuA.jpeg
paper_authors.fullname: André Storhaug
paper_authors.type: user
paper_authors.user: andstor
paper_authors._id: 673171234edd034cb4bdaf74
paper_authors.name: Jingyue Li
paper_title: Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test
    Generation: An Empirical Study
paper_summary: The advent of large language models (LLMs) like GitHub Copilot has
  significantly enhanced programmers' productivity, particularly in code
  generation. However, these models often struggle with real-world tasks without
  fine-tuning. As LLMs grow larger and more performant, fine-tuning for
  specialized tasks becomes increasingly expensive. Parameter-efficient
  fine-tuning (PEFT) methods, which fine-tune only a subset of model parameters,
  offer a promising solution by reducing the computational costs of tuning LLMs
  while maintaining their performance. Existing studies have explored using PEFT
  and LLMs for various code-related tasks and found that the effectiveness of
  PEFT techniques is task-dependent. The application of PEFT techniques in unit
  test generation remains underexplored. The state-of-the-art is limited to using
  LLMs with full fine-tuning to generate unit tests. This paper investigates both
  full fine-tuning and various PEFT methods, including LoRA, (IA)^3, and prompt
  tuning, across different model architectures and sizes. We use well-established
  benchmark datasets to evaluate their effectiveness in unit test generation. Our
  findings show that PEFT methods can deliver performance comparable to full
  fine-tuning for unit test generation, making specialized fine-tuning more
  accessible and cost-effective. Notably, prompt tuning is the most effective in
  terms of cost and resource utilization, while LoRA approaches the effectiveness
  of full fine-tuning in several cases.
paper_ai_summary: Research investigates the effectiveness of parameter-efficient fine-tuning methods compared to full fine-tuning for generating unit tests using large language models.
paper_ai_keywords: large language models LLMs GitHub Copilot fine-tuning parameter-efficient fine-tuning PEFT LoRA (IA)^3 prompt tuning unit test generation benchmark datasets