title: CODESIM: Multi-Agent Code Generation and Problem Solving through
    Simulation-Driven Planning and Debugging
summary: Large Language Models (LLMs) have made significant strides in code generation
  and problem solving. Current approaches employ external tool-based iterative
  debuggers that use compiler or other tool-based runtime feedback to refine
  coarse programs generated by various methods. However, the effectiveness of
  these approaches heavily relies on the quality of the initial code generation,
  which remains an open challenge. In this paper, we introduce CodeSim, a novel
  multi-agent code generation framework that comprehensively addresses the stages
  of program synthesis-planning, coding, and debugging-through a human-like
  perception approach. As human verifies their understanding of any algorithms
  through visual simulation, CodeSim uniquely features a method of plan
  verification and internal debugging through the step-by-step simulation of
  input/output. Extensive experiments across seven challenging competitive
  problem-solving and program synthesis benchmarks demonstrate CodeSim's
  remarkable code generation capabilities. Our framework achieves new
  state-of-the-art (pass@1) results-(HumanEval 95.1%, MBPP 90.7%, APPS 22%, and
  CodeContests 29.1%). Furthermore, our method shows potential for even greater
  enhancement when cascaded with external debuggers. To facilitate further
  research and development in this area, we have open-sourced our framework in
  this link (https://kagnlp.github.io/codesim.github.io/).
paper_authors._id: 67ab56dc0bc5f6a94eb49892
paper_authors.name: Md. Ashraful Islam
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-11T14:25:21.564Z
paper_authors._id: 63fe51dcc0ec83fda436d558
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63fe51dcc0ec83fda436d558/22wrFA08OxRLIsRVxPts0.jpeg
paper_authors.fullname: Md. Ashraful Islam
paper_authors.type: user
paper_authors.user: ashraful
paper_authors._id: 67ab56dc0bc5f6a94eb49893
paper_authors.name: Mohammed Eunus Ali
paper_authors._id: 67ab56dc0bc5f6a94eb49894
paper_authors.name: Md Rizwan Parvez
paper_title: CODESIM: Multi-Agent Code Generation and Problem Solving through
    Simulation-Driven Planning and Debugging
paper_summary: Large Language Models (LLMs) have made significant strides in code generation
  and problem solving. Current approaches employ external tool-based iterative
  debuggers that use compiler or other tool-based runtime feedback to refine
  coarse programs generated by various methods. However, the effectiveness of
  these approaches heavily relies on the quality of the initial code generation,
  which remains an open challenge. In this paper, we introduce CodeSim, a novel
  multi-agent code generation framework that comprehensively addresses the stages
  of program synthesis-planning, coding, and debugging-through a human-like
  perception approach. As human verifies their understanding of any algorithms
  through visual simulation, CodeSim uniquely features a method of plan
  verification and internal debugging through the step-by-step simulation of
  input/output. Extensive experiments across seven challenging competitive
  problem-solving and program synthesis benchmarks demonstrate CodeSim's
  remarkable code generation capabilities. Our framework achieves new
  state-of-the-art (pass@1) results-(HumanEval 95.1%, MBPP 90.7%, APPS 22%, and
  CodeContests 29.1%). Furthermore, our method shows potential for even greater
  enhancement when cascaded with external debuggers. To facilitate further
  research and development in this area, we have open-sourced our framework in
  this link (https://kagnlp.github.io/codesim.github.io/).
paper_ai_keywords: Large Language Models (LLMs) code generation problem solving external tool-based iterative debuggers compiler runtime feedback program synthesis planning coding debugging human-like perception plan verification internal debugging step-by-step simulation input/output HumanEval MBPP APPS CodeContests pass@1