title: From Code to Correctness: Closing the Last Mile of Code Generation with
    Hierarchical Debugging
summary: While large language models have made significant strides in code generation,
  the pass rate of the generated code is bottlenecked on subtle errors, often
  requiring human intervention to pass tests, especially for complex problems.
  Existing LLM-based debugging systems treat generated programs as monolithic
  units, failing to address bugs at multiple levels of granularity, from
  low-level syntax errors to high-level algorithmic flaws. In this paper, we
  introduce Multi-Granularity Debugger (MGDebugger), a hierarchical code debugger
  by isolating, identifying, and resolving bugs at various levels of granularity.
  MGDebugger decomposes problematic code into a hierarchical tree structure of
  subfunctions, with each level representing a particular granularity of error.
  During debugging, it analyzes each subfunction and iteratively resolves bugs in
  a bottom-up manner. To effectively test each subfunction, we propose an
  LLM-simulated Python executor, which traces code execution and tracks important
  variable states to pinpoint errors accurately. Extensive experiments
  demonstrate that MGDebugger outperforms existing debugging systems, achieving
  an 18.9% improvement in accuracy over seed generations in HumanEval and a 97.6%
  repair success rate in HumanEvalFix. Furthermore, MGDebugger effectively fixes
  bugs across different categories and difficulty levels, demonstrating its
  robustness and effectiveness.
paper_authors._id: 66fe140c7d722f0879868725
paper_authors.name: Yuling Shi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-03T08:31:20.232Z
paper_authors._id: 645b0c3ec35da9c7afd95421
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645b0c3ec35da9c7afd95421/vYBrCDagHsXAo6J2p-uG0.jpeg
paper_authors.fullname: Yuling
paper_authors.type: user
paper_authors.user: YerbaPage
paper_authors._id: 66fe140c7d722f0879868726
paper_authors.name: Songsong Wang
paper_authors._id: 66fe140c7d722f0879868727
paper_authors.name: Chengcheng Wan
paper_authors._id: 66fe140c7d722f0879868728
paper_authors.name: Xiaodong Gu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-07T07:28:40.543Z
paper_authors._id: 6624ede5478dada13051af60
paper_authors.avatarUrl: /avatars/1b064f7e3c417f1b603909ff3dbf2430.svg
paper_authors.fullname: xiaodong gu
paper_authors.type: user
paper_authors.user: guxiaodong1987
paper_title: From Code to Correctness: Closing the Last Mile of Code Generation with
    Hierarchical Debugging
paper_summary: While large language models have made significant strides in code generation,
  the pass rate of the generated code is bottlenecked on subtle errors, often
  requiring human intervention to pass tests, especially for complex problems.
  Existing LLM-based debugging systems treat generated programs as monolithic
  units, failing to address bugs at multiple levels of granularity, from
  low-level syntax errors to high-level algorithmic flaws. In this paper, we
  introduce Multi-Granularity Debugger (MGDebugger), a hierarchical code debugger
  by isolating, identifying, and resolving bugs at various levels of granularity.
  MGDebugger decomposes problematic code into a hierarchical tree structure of
  subfunctions, with each level representing a particular granularity of error.
  During debugging, it analyzes each subfunction and iteratively resolves bugs in
  a bottom-up manner. To effectively test each subfunction, we propose an
  LLM-simulated Python executor, which traces code execution and tracks important
  variable states to pinpoint errors accurately. Extensive experiments
  demonstrate that MGDebugger outperforms existing debugging systems, achieving
  an 18.9% improvement in accuracy over seed generations in HumanEval and a 97.6%
  repair success rate in HumanEvalFix. Furthermore, MGDebugger effectively fixes
  bugs across different categories and difficulty levels, demonstrating its
  robustness and effectiveness.
paper_ai_keywords: Large language models (LLM) Code generation Hierarchical code debugger Subfunctions Bottom-up manner LLM-simulated Python executor Code execution Variable states HumanEval HumanEvalFix