Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Code
<|reference_start|>Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Code: This paper presents Coffee-Gym, a comprehensive RL environment for training models that provide feedback on code editing. Coffee-Gym includes two major components: (1) Coffee, a dataset containing humans' code edit traces for coding questions and machine-written feedback for editing erroneous code; (2) CoffeeEval, a reward function that faithfully reflects the helpfulness of feedback by assessing the performance of the revised code in unit tests. With them, Coffee-Gym addresses the unavailability of high-quality datasets for training feedback models with RL, and provides more accurate rewards than the SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback models that outperform baselines in enhancing open-source code LLMs' code editing, making them comparable with closed-source LLMs. We make the dataset and the model checkpoint publicly available.<|reference_end|>
@article{chae2024coffee-gym:,
    title={Coffee-Gym: An Environment for Evaluating and Improving Natural Language
  Feedback on Erroneous Code},
    author={Hyungjoo Chae, Taeyoon Kwon, Seungjun Moon, Yongho Song, Dongjin Kang,
  Kai Tzu-iunn Ong, Beong-woo Kwak, Seonghyeon Bae, Seung-won Hwang, Jinyoung
  Yeo},
    journal={arXiv preprint arXiv:2409.19715},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.19715},
    primaryClass={cs.CL}
}