title: How Far Can We Go with Practical Function-Level Program Repair?
summary: Recently, multiple Automated Program Repair (APR) techniques based on Large
  Language Models (LLMs) have been proposed to enhance the repair performance.
  While these techniques mainly focus on the single-line or hunk-level repair,
  they face significant challenges in real-world application due to the limited
  repair task scope and costly statement-level fault localization. However, the
  more practical function-level APR, which broadens the scope of APR task to fix
  entire buggy functions and requires only cost-efficient function-level fault
  localization, remains underexplored. In this paper, we conduct the first
  comprehensive study of LLM-based function-level APR including investigating the
  effect of the few-shot learning mechanism and the auxiliary repair-relevant
  information. Specifically, we adopt six widely-studied LLMs and construct a
  benchmark in both the Defects4J 1.2 and 2.0 datasets. Our study demonstrates
  that LLMs with zero-shot learning are already powerful function-level APR
  techniques, while applying the few-shot learning mechanism leads to disparate
  repair performance. Moreover, we find that directly applying the auxiliary
  repair-relevant information to LLMs significantly increases function-level
  repair performance. Inspired by our findings, we propose an LLM-based
  function-level APR technique, namely SRepair, which adopts a dual-LLM framework
  to leverage the power of the auxiliary repair-relevant information for
  advancing the repair performance. The evaluation results demonstrate that
  SRepair can correctly fix 300 single-function bugs in the Defects4J dataset,
  largely surpassing all previous APR techniques by at least 85%, without the
  need for the costly statement-level fault location information. Furthermore,
  SRepair successfully fixes 32 multi-function bugs in the Defects4J dataset,
  which is the first time achieved by any APR technique ever to our best
  knowledge.
paper_authors._id: 6625f4a026089aaae05d5978
paper_authors.name: Jiahong Xiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-23T07:15:24.793Z
paper_authors._id: 662673feea153ffe143eb154
paper_authors.avatarUrl: /avatars/416127c5fd920d692bbe862bc3344a73.svg
paper_authors.fullname: Jiahong Xiang
paper_authors.type: user
paper_authors.user: GhabiX
paper_authors._id: 6625f4a026089aaae05d5979
paper_authors.name: Xiaoyang Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-22T14:07:33.668Z
paper_authors._id: 6501215a85a884a964dc0ded
paper_authors.avatarUrl: /avatars/1ca84b3d19d4ac4da9c67ca1627bf186.svg
paper_authors.fullname: xiaoyang xu
paper_authors.type: user
paper_authors.user: xuxy
paper_authors._id: 6625f4a026089aaae05d597a
paper_authors.name: Fanchu Kong
paper_authors._id: 6625f4a026089aaae05d597b
paper_authors.name: Mingyuan Wu
paper_authors._id: 6625f4a026089aaae05d597c
paper_authors.name: Haotian Zhang
paper_authors._id: 6625f4a026089aaae05d597d
paper_authors.name: Yuqun Zhang
paper_title: How Far Can We Go with Practical Function-Level Program Repair?
paper_summary: Recently, multiple Automated Program Repair (APR) techniques based on Large
  Language Models (LLMs) have been proposed to enhance the repair performance.
  While these techniques mainly focus on the single-line or hunk-level repair,
  they face significant challenges in real-world application due to the limited
  repair task scope and costly statement-level fault localization. However, the
  more practical function-level APR, which broadens the scope of APR task to fix
  entire buggy functions and requires only cost-efficient function-level fault
  localization, remains underexplored. In this paper, we conduct the first
  comprehensive study of LLM-based function-level APR including investigating the
  effect of the few-shot learning mechanism and the auxiliary repair-relevant
  information. Specifically, we adopt six widely-studied LLMs and construct a
  benchmark in both the Defects4J 1.2 and 2.0 datasets. Our study demonstrates
  that LLMs with zero-shot learning are already powerful function-level APR
  techniques, while applying the few-shot learning mechanism leads to disparate
  repair performance. Moreover, we find that directly applying the auxiliary
  repair-relevant information to LLMs significantly increases function-level
  repair performance. Inspired by our findings, we propose an LLM-based
  function-level APR technique, namely SRepair, which adopts a dual-LLM framework
  to leverage the power of the auxiliary repair-relevant information for
  advancing the repair performance. The evaluation results demonstrate that
  SRepair can correctly fix 300 single-function bugs in the Defects4J dataset,
  largely surpassing all previous APR techniques by at least 85%, without the
  need for the costly statement-level fault location information. Furthermore,
  SRepair successfully fixes 32 multi-function bugs in the Defects4J dataset,
  which is the first time achieved by any APR technique ever to our best
  knowledge.
paper_ai_summary: A study reveals that LLMs, particularly those using zero-shot learning and auxiliary repair information, outperform traditional APR techniques in function-level repairs, especially when using a dual-LLM framework.
paper_ai_keywords: Large Language Models Automated Program Repair few-shot learning zero-shot learning Defects4J auxiliary repair-relevant information dual-LLM framework