OpenCodeInterpreter: Integrating Code Generation with Execution and
  Refinement
The introduction of large language models has significantly advanced code
generation. However, open-source models often lack the execution capabilities
and iterative refinement of advanced systems like the GPT-4 Code Interpreter.
To address this, we introduce OpenCodeInterpreter, a family of open-source code
systems designed for generating, executing, and iteratively refining code.
Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions,
OpenCodeInterpreter integrates execution and human feedback for dynamic code
refinement. Our comprehensive evaluation of OpenCodeInterpreter across key
benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus
reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves
an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and
MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6)
with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap
between open-source code generation models and proprietary systems like GPT-4
Code Interpreter.
65d80b1e6769b3a9c92181e7
Tianyu Zheng
claimed_verified
2024-02-23T12:25:15.765Z
64ab99dcb76bfd863eba64c1
https://cdn-avatars.huggingface.co/v1/production/uploads/64ab99dcb76bfd863eba64c1/UBXwDPx17X-gl-SzBPvrc.jpeg
TY.Zheng
user
aaabiao
65d80b1e6769b3a9c92181e8
Ge Zhang
admin_assigned
2024-02-23T12:30:53.939Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
65d80b1e6769b3a9c92181e9
Tianhao Shen
admin_assigned
2024-02-23T12:28:49.969Z
61976063b1099a2392101960
https://cdn-avatars.huggingface.co/v1/production/uploads/61976063b1099a2392101960/DRlAoI6cD6yyomQfWRTU5.png
Tianhao Shen
user
magnetic
65d80b1e6769b3a9c92181ea
Xueling Liu
admin_assigned
2024-02-23T12:29:09.748Z
65865ee6a66bd1cdb22d1adb
/avatars/97777298239e505fea4e96577b89eef8.svg
XUELING LIU
user
Anitaliu98
65d80b1e6769b3a9c92181eb
Bill Yuchen Lin
admin_assigned
2024-02-23T12:29:16.074Z
607f666a4ad99100d63ce35c
https://cdn-avatars.huggingface.co/v1/production/uploads/607f666a4ad99100d63ce35c/QxhxnvfeV6efkxwUFHwjI.png
Bill Yuchen Lin
user
yuchenlin
65d80b1e6769b3a9c92181ec
Jie Fu
admin_assigned
2024-02-23T12:29:32.889Z
641a6895fb5ffff5ac79d593
https://cdn-avatars.huggingface.co/v1/production/uploads/641a6895fb5ffff5ac79d593/vxvwsto3llOEWGqQKGMYx.jpeg
Jie Fu
user
bigaidream
65d80b1e6769b3a9c92181ed
Wenhu Chen
admin_assigned
2024-02-23T12:29:38.947Z
6313a86154e6e5d9f0f94e04
https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
Wenhu Chen
user
wenhu
65d80b1e6769b3a9c92181ee
Xiang Yue
admin_assigned
2024-02-23T12:29:57.861Z
6230d750d93e84e233882dbc
https://cdn-avatars.huggingface.co/v1/production/uploads/6230d750d93e84e233882dbc/4MGEekLW3oWzqeFWDWvIK.jpeg
Xiang Yue
user
yuexiang96
OpenCodeInterpreter: Integrating Code Generation with Execution and
  Refinement
The introduction of large language models has significantly advanced code
generation. However, open-source models often lack the execution capabilities
and iterative refinement of advanced systems like the GPT-4 Code Interpreter.
To address this, we introduce OpenCodeInterpreter, a family of open-source code
systems designed for generating, executing, and iteratively refining code.
Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions,
OpenCodeInterpreter integrates execution and human feedback for dynamic code
refinement. Our comprehensive evaluation of OpenCodeInterpreter across key
benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus
reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves
an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and
MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6)
with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap
between open-source code generation models and proprietary systems like GPT-4
Code Interpreter.
OpenCodeInterpreter, an open-source system for generating, executing, and refining code, achieves high performance on benchmarks through execution and human feedback, reducing the gap with proprietary systems like GPT-4 Code Interpreter.
large language models code generation execution capabilities iterative refinement code systems Code-Feedback dataset dynamic code refinement HumanEval MBPP EvalPlus benchmark evaluation accuracy