title: CodePlan: Repository-level Coding using LLMs and Planning
summary: Software engineering activities such as package migration, fixing errors
  reports from static analysis or testing, and adding type annotations or other
  specifications to a codebase, involve pervasively editing the entire repository
  of code. We formulate these activities as repository-level coding tasks.
    Recent tools like GitHub Copilot, which are powered by Large Language Models
  (LLMs), have succeeded in offering high-quality solutions to localized coding
  problems. Repository-level coding tasks are more involved and cannot be solved
  directly using LLMs, since code within a repository is inter-dependent and the
  entire repository may be too large to fit into the prompt. We frame
  repository-level coding as a planning problem and present a task-agnostic
  framework, called CodePlan to solve it. CodePlan synthesizes a multi-step chain
  of edits (plan), where each step results in a call to an LLM on a code location
  with context derived from the entire repository, previous code changes and
  task-specific instructions. CodePlan is based on a novel combination of an
  incremental dependency analysis, a change may-impact analysis and an adaptive
  planning algorithm.
    We evaluate the effectiveness of CodePlan on two repository-level tasks:
  package migration (C#) and temporal code edits (Python). Each task is evaluated
  on multiple code repositories, each of which requires inter-dependent changes
  to many files (between 2-97 files). Coding tasks of this level of complexity
  have not been automated using LLMs before. Our results show that CodePlan has
  better match with the ground truth compared to baselines. CodePlan is able to
  get 5/6 repositories to pass the validity checks (e.g., to build without errors
  and make correct code edits) whereas the baselines (without planning but with
  the same type of contextual information as CodePlan) cannot get any of the
  repositories to pass them.
paper_authors._id: 6510e1dd11f3210cf7da75e4
paper_authors.name: Ramakrishna Bairi
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-09-25T12:37:24.824Z
paper_authors._id: 65117dfad4556f9e65dd9589
paper_authors.avatarUrl: /avatars/bbe19aae13b2125688ac57bd830128fd.svg
paper_authors.fullname: Ramakrishna B Bairi
paper_authors.type: user
paper_authors.user: rkbairi
paper_authors._id: 6510e1dd11f3210cf7da75e5
paper_authors.name: Atharv Sonwane
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-25T10:54:17.381Z
paper_authors._id: 64c7c14daf935d3927b99dba
paper_authors.avatarUrl: /avatars/704e85446e9908ddb00aea160096b21b.svg
paper_authors.fullname: Atharv Sonwane
paper_authors.type: user
paper_authors.user: twm-as
paper_authors._id: 6510e1dd11f3210cf7da75e6
paper_authors.name: Aditya Kanade
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-25T10:54:33.126Z
paper_authors._id: 649e62e953777d8950749c72
paper_authors.avatarUrl: /avatars/980c04bbbe4e18b2a364f597fe60ba60.svg
paper_authors.fullname: AK
paper_authors.type: user
paper_authors.user: adityakanade
paper_authors._id: 6510e1dd11f3210cf7da75e7
paper_authors.name: Vageesh D C
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-25T10:54:42.752Z
paper_authors._id: 63916c4b239695d224083de6
paper_authors.avatarUrl: /avatars/45f44cea1b313c01f2dec9bb625633c6.svg
paper_authors.fullname: .
paper_authors.type: user
paper_authors.user: Vageeshdc
paper_authors._id: 6510e1dd11f3210cf7da75e8
paper_authors.name: Arun Iyer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-25T10:54:50.263Z
paper_authors._id: 6493172227ebd33660c65f3b
paper_authors.avatarUrl: /avatars/eada49ee732b78b073d4fcae252bd875.svg
paper_authors.fullname: Arun Iyer
paper_authors.type: user
paper_authors.user: tenaliraman
paper_authors._id: 6510e1dd11f3210cf7da75e9
paper_authors.name: Suresh Parthasarathy
paper_authors._id: 6510e1dd11f3210cf7da75ea
paper_authors.name: Sriram Rajamani
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-09-25T01:26:56.590Z
paper_authors._id: 63fb54480aab060792f71151
paper_authors.avatarUrl: /avatars/97edbaed65e87a555bf05f770fa60b23.svg
paper_authors.fullname: Sriram Rajamani
paper_authors.type: user
paper_authors.user: sriramkr
paper_authors._id: 6510e1dd11f3210cf7da75eb
paper_authors.name: B. Ashok
paper_authors._id: 6510e1dd11f3210cf7da75ec
paper_authors.name: Shashank Shet
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-25T10:55:23.484Z
paper_authors._id: 64c5f012589826641baf1f6f
paper_authors.avatarUrl: /avatars/5b059e33d95a42ce219f03a00d11ee2a.svg
paper_authors.fullname: Shashank Shet
paper_authors.type: user
paper_authors.user: shashankshet
paper_title: CodePlan: Repository-level Coding using LLMs and Planning
paper_summary: Software engineering activities such as package migration, fixing errors
  reports from static analysis or testing, and adding type annotations or other
  specifications to a codebase, involve pervasively editing the entire repository
  of code. We formulate these activities as repository-level coding tasks.
    Recent tools like GitHub Copilot, which are powered by Large Language Models
  (LLMs), have succeeded in offering high-quality solutions to localized coding
  problems. Repository-level coding tasks are more involved and cannot be solved
  directly using LLMs, since code within a repository is inter-dependent and the
  entire repository may be too large to fit into the prompt. We frame
  repository-level coding as a planning problem and present a task-agnostic
  framework, called CodePlan to solve it. CodePlan synthesizes a multi-step chain
  of edits (plan), where each step results in a call to an LLM on a code location
  with context derived from the entire repository, previous code changes and
  task-specific instructions. CodePlan is based on a novel combination of an
  incremental dependency analysis, a change may-impact analysis and an adaptive
  planning algorithm.
    We evaluate the effectiveness of CodePlan on two repository-level tasks:
  package migration (C#) and temporal code edits (Python). Each task is evaluated
  on multiple code repositories, each of which requires inter-dependent changes
  to many files (between 2-97 files). Coding tasks of this level of complexity
  have not been automated using LLMs before. Our results show that CodePlan has
  better match with the ground truth compared to baselines. CodePlan is able to
  get 5/6 repositories to pass the validity checks (e.g., to build without errors
  and make correct code edits) whereas the baselines (without planning but with
  the same type of contextual information as CodePlan) cannot get any of the
  repositories to pass them.
paper_ai_keywords: GitHub Copilot Large Language Models (LLMs) repository-level coding tasks planning problem incremental dependency analysis change may-impact analysis adaptive planning algorithm package migration temporal code edits