title: OpenCodeInterpreter: Integrating Code Generation with Execution and
    Refinement
summary: The introduction of large language models has significantly advanced code
  generation. However, open-source models often lack the execution capabilities
  and iterative refinement of advanced systems like the GPT-4 Code Interpreter.
  To address this, we introduce OpenCodeInterpreter, a family of open-source code
  systems designed for generating, executing, and iteratively refining code.
  Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions,
  OpenCodeInterpreter integrates execution and human feedback for dynamic code
  refinement. Our comprehensive evaluation of OpenCodeInterpreter across key
  benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus
  reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves
  an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and
  MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6)
  with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap
  between open-source code generation models and proprietary systems like GPT-4
  Code Interpreter.
paper_authors._id: 65d80b1e6769b3a9c92181e7
paper_authors.name: Tianyu Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-23T12:25:15.765Z
paper_authors._id: 64ab99dcb76bfd863eba64c1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ab99dcb76bfd863eba64c1/UBXwDPx17X-gl-SzBPvrc.jpeg
paper_authors.fullname: TY.Zheng
paper_authors.type: user
paper_authors.user: aaabiao
paper_authors._id: 65d80b1e6769b3a9c92181e8
paper_authors.name: Ge Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:30:53.939Z
paper_authors._id: 638efcf4c67af472d316d424
paper_authors.avatarUrl: /avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
paper_authors.fullname: Ge Zhang
paper_authors.type: user
paper_authors.user: zhangysk
paper_authors._id: 65d80b1e6769b3a9c92181e9
paper_authors.name: Tianhao Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:28:49.969Z
paper_authors._id: 61976063b1099a2392101960
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61976063b1099a2392101960/DRlAoI6cD6yyomQfWRTU5.png
paper_authors.fullname: Tianhao Shen
paper_authors.type: user
paper_authors.user: magnetic
paper_authors._id: 65d80b1e6769b3a9c92181ea
paper_authors.name: Xueling Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:29:09.748Z
paper_authors._id: 65865ee6a66bd1cdb22d1adb
paper_authors.avatarUrl: /avatars/97777298239e505fea4e96577b89eef8.svg
paper_authors.fullname: XUELING LIU
paper_authors.type: user
paper_authors.user: Anitaliu98
paper_authors._id: 65d80b1e6769b3a9c92181eb
paper_authors.name: Bill Yuchen Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:29:16.074Z
paper_authors._id: 607f666a4ad99100d63ce35c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/607f666a4ad99100d63ce35c/QxhxnvfeV6efkxwUFHwjI.png
paper_authors.fullname: Bill Yuchen Lin
paper_authors.type: user
paper_authors.user: yuchenlin
paper_authors._id: 65d80b1e6769b3a9c92181ec
paper_authors.name: Jie Fu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:29:32.889Z
paper_authors._id: 641a6895fb5ffff5ac79d593
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/641a6895fb5ffff5ac79d593/dFR_ofjbqCrcqGa9R3MMq.jpeg
paper_authors.fullname: Jie Fu
paper_authors.type: user
paper_authors.user: bigaidream
paper_authors._id: 65d80b1e6769b3a9c92181ed
paper_authors.name: Wenhu Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:29:38.947Z
paper_authors._id: 6313a86154e6e5d9f0f94e04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
paper_authors.fullname: Wenhu Chen
paper_authors.type: user
paper_authors.user: wenhu
paper_authors._id: 65d80b1e6769b3a9c92181ee
paper_authors.name: Xiang Yue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:29:57.861Z
paper_authors._id: 6230d750d93e84e233882dbc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6230d750d93e84e233882dbc/4MGEekLW3oWzqeFWDWvIK.jpeg
paper_authors.fullname: Xiang Yue
paper_authors.type: user
paper_authors.user: yuexiang96
paper_title: OpenCodeInterpreter: Integrating Code Generation with Execution and
    Refinement
paper_summary: The introduction of large language models has significantly advanced code
  generation. However, open-source models often lack the execution capabilities
  and iterative refinement of advanced systems like the GPT-4 Code Interpreter.
  To address this, we introduce OpenCodeInterpreter, a family of open-source code
  systems designed for generating, executing, and iteratively refining code.
  Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions,
  OpenCodeInterpreter integrates execution and human feedback for dynamic code
  refinement. Our comprehensive evaluation of OpenCodeInterpreter across key
  benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus
  reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves
  an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and
  MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6)
  with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap
  between open-source code generation models and proprietary systems like GPT-4
  Code Interpreter.
paper_ai_keywords: large language models code generation GPT-4 Code Interpreter OpenCodeInterpreter Code-Feedback HumanEval MBPP dynamic code refinement synthesized human feedback