Coffee-Gym: An Environment for Evaluating and Improving Natural Language
  Feedback on Erroneous Code
This paper presents Coffee-Gym, a comprehensive RL environment for training
models that provide feedback on code editing. Coffee-Gym includes two major
components: (1) Coffee, a dataset containing humans' code edit traces for
coding questions and machine-written feedback for editing erroneous code; (2)
CoffeeEval, a reward function that faithfully reflects the helpfulness of
feedback by assessing the performance of the revised code in unit tests. With
them, Coffee-Gym addresses the unavailability of high-quality datasets for
training feedback models with RL, and provides more accurate rewards than the
SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback
models that outperform baselines in enhancing open-source code LLMs' code
editing, making them comparable with closed-source LLMs. We make the dataset
and the model checkpoint publicly available.
66fbbcffc247c5b3c24ee5d1
Hyungjoo Chae
admin_assigned
2024-10-01T13:24:26.737Z
64c8f4cec547ed5243ebd0a8
https://cdn-avatars.huggingface.co/v1/production/uploads/64c8f4cec547ed5243ebd0a8/MiOH5YbMg8Gh9KYlQsLmX.jpeg
Hyungjoo Chae
user
hyungjoochae
66fbbcffc247c5b3c24ee5d2
Taeyoon Kwon
admin_assigned
2024-10-01T13:24:35.670Z
636b529ef796304dd67d139c
/avatars/7a64d5095fcb1da558b52ad48177ad76.svg
Taeyoon Kwon
user
Connoriginal
66fbbcffc247c5b3c24ee5d3
Seungjun Moon
admin_assigned
2024-10-01T13:24:42.007Z
6420f4f55bccaa42484496e5
/avatars/4996ba26955f8423c946b1ecd3989964.svg
Seung Jun Moon
user
Lune-Blue
66fbbcffc247c5b3c24ee5d4
Yongho Song
admin_assigned
2024-10-01T13:24:47.537Z
64c92f1a5381684d3eb1b0d1
/avatars/35e500bcc486a274dbb9a420f4b17830.svg
Yongho Song
user
yonghosong
66fbbcffc247c5b3c24ee5d5
Dongjin Kang
66fbbcffc247c5b3c24ee5d6
Kai Tzu-iunn Ong
admin_assigned
2024-10-01T13:25:06.059Z
640ec2fd2f9c7b364d182735
https://cdn-avatars.huggingface.co/v1/production/uploads/640ec2fd2f9c7b364d182735/G5WxQXYEOaVL39rAoOI3Y.jpeg
kai tzu-iunn ong
user
ktio
66fbbcffc247c5b3c24ee5d7
Beong-woo Kwak
admin_assigned
2024-10-01T13:25:13.937Z
64b72a408ba7d6c922c73054
/avatars/6d9797430bc36f05fb950b84aa6a9374.svg
Beong Woo Kwak
user
bwookwak
66fbbcffc247c5b3c24ee5d8
Seonghyeon Bae
66fbbcffc247c5b3c24ee5d9
Seung-won Hwang
66fbbcffc247c5b3c24ee5da
Jinyoung Yeo
Coffee-Gym: An Environment for Evaluating and Improving Natural Language
  Feedback on Erroneous Code
This paper presents Coffee-Gym, a comprehensive RL environment for training
models that provide feedback on code editing. Coffee-Gym includes two major
components: (1) Coffee, a dataset containing humans' code edit traces for
coding questions and machine-written feedback for editing erroneous code; (2)
CoffeeEval, a reward function that faithfully reflects the helpfulness of
feedback by assessing the performance of the revised code in unit tests. With
them, Coffee-Gym addresses the unavailability of high-quality datasets for
training feedback models with RL, and provides more accurate rewards than the
SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback
models that outperform baselines in enhancing open-source code LLMs' code
editing, making them comparable with closed-source LLMs. We make the dataset
and the model checkpoint publicly available.
Coffee-Gym, an RL environment using a human code edit dataset and a performance-based reward function, trains feedback models that enhance open-source code LLMs' code editing abilities.
RL environment code edit traces machine-written feedback reward function unit tests feedback models open-source code LLMs closed-source LLMs dataset model checkpoint