title: RLTF: Reinforcement Learning from Unit Test Feedback
summary: The goal of program synthesis, or code generation, is to generate executable
  code based on given descriptions. Recently, there has been an increasing number
  of studies employing reinforcement learning (RL) to improve the performance of
  large language models (LLMs) for code. However, these RL methods have only used
  offline frameworks, limiting their exploration of new sample spaces.
  Additionally, current approaches that utilize unit test signals are rather
  simple, not accounting for specific error locations within the code. To address
  these issues, we proposed RLTF, i.e., Reinforcement Learning from Unit Test
  Feedback, a novel online RL framework with unit test feedback of
  multi-granularity for refining code LLMs. Our approach generates data in
  real-time during training and simultaneously utilizes fine-grained feedback
  signals to guide the model towards producing higher-quality code. Extensive
  experiments show that RLTF achieves state-of-the-art performance on the APPS
  and the MBPP benchmarks. Our code can be found at:
  https://github.com/Zyq-scut/RLTF.
paper_authors._id: 64acb35058bd9e9cc2c1b2fc
paper_authors.name: Jiate Liu
paper_authors._id: 64acb35058bd9e9cc2c1b2fd
paper_authors.name: Yiqin Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-12T07:29:49.317Z
paper_authors._id: 63e3924c19b1b962a43e331c
paper_authors.avatarUrl: /avatars/b7b741bec01b7479a16e5e17dcef472d.svg
paper_authors.fullname: yiqin zhu
paper_authors.type: user
paper_authors.user: Zyego
paper_authors._id: 64acb35058bd9e9cc2c1b2fe
paper_authors.name: Kaiwen Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-18T06:44:55.326Z
paper_authors._id: 6441491c5d600fb0951cd872
paper_authors.avatarUrl: /avatars/d98892f3b52d87c2328201efa9897110.svg
paper_authors.fullname: Kaiwen Xiao
paper_authors.type: user
paper_authors.user: loktarxiao
paper_authors._id: 64acb35058bd9e9cc2c1b2ff
paper_authors.name: Qiang Fu
paper_authors._id: 64acb35058bd9e9cc2c1b300
paper_authors.name: Xiao Han
paper_authors._id: 64acb35058bd9e9cc2c1b301
paper_authors.name: Wei Yang
paper_authors._id: 64acb35058bd9e9cc2c1b302
paper_authors.name: Deheng Ye
paper_title: RLTF: Reinforcement Learning from Unit Test Feedback
paper_summary: The goal of program synthesis, or code generation, is to generate executable
  code based on given descriptions. Recently, there has been an increasing number
  of studies employing reinforcement learning (RL) to improve the performance of
  large language models (LLMs) for code. However, these RL methods have only used
  offline frameworks, limiting their exploration of new sample spaces.
  Additionally, current approaches that utilize unit test signals are rather
  simple, not accounting for specific error locations within the code. To address
  these issues, we proposed RLTF, i.e., Reinforcement Learning from Unit Test
  Feedback, a novel online RL framework with unit test feedback of
  multi-granularity for refining code LLMs. Our approach generates data in
  real-time during training and simultaneously utilizes fine-grained feedback
  signals to guide the model towards producing higher-quality code. Extensive
  experiments show that RLTF achieves state-of-the-art performance on the APPS
  and the MBPP benchmarks. Our code can be found at:
  https://github.com/Zyq-scut/RLTF.
paper_ai_keywords: reinforcement learning (RL) large language models (LLMs) code synthesis offline frameworks unit test signals RLTF Reinforcement Learning from Unit Test Feedback multi-granularity feedback real-time data generation fine-grained feedback signals APPS benchmark MBPP benchmark