title: Coffee-Gym: An Environment for Evaluating and Improving Natural Language
    Feedback on Erroneous Code
summary: This paper presents Coffee-Gym, a comprehensive RL environment for training
  models that provide feedback on code editing. Coffee-Gym includes two major
  components: (1) Coffee, a dataset containing humans' code edit traces for
  coding questions and machine-written feedback for editing erroneous code; (2)
  CoffeeEval, a reward function that faithfully reflects the helpfulness of
  feedback by assessing the performance of the revised code in unit tests. With
  them, Coffee-Gym addresses the unavailability of high-quality datasets for
  training feedback models with RL, and provides more accurate rewards than the
  SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback
  models that outperform baselines in enhancing open-source code LLMs' code
  editing, making them comparable with closed-source LLMs. We make the dataset
  and the model checkpoint publicly available.
paper_authors._id: 66fbbcffc247c5b3c24ee5d1
paper_authors.name: Hyungjoo Chae
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-01T13:24:26.737Z
paper_authors._id: 64c8f4cec547ed5243ebd0a8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c8f4cec547ed5243ebd0a8/MiOH5YbMg8Gh9KYlQsLmX.jpeg
paper_authors.fullname: Hyungjoo Chae
paper_authors.type: user
paper_authors.user: hyungjoochae
paper_authors._id: 66fbbcffc247c5b3c24ee5d2
paper_authors.name: Taeyoon Kwon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-01T13:24:35.670Z
paper_authors._id: 636b529ef796304dd67d139c
paper_authors.avatarUrl: /avatars/7a64d5095fcb1da558b52ad48177ad76.svg
paper_authors.fullname: Taeyoon Kwon
paper_authors.type: user
paper_authors.user: Connoriginal
paper_authors._id: 66fbbcffc247c5b3c24ee5d3
paper_authors.name: Seungjun Moon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-01T13:24:42.007Z
paper_authors._id: 6420f4f55bccaa42484496e5
paper_authors.avatarUrl: /avatars/4996ba26955f8423c946b1ecd3989964.svg
paper_authors.fullname: Seung Jun Moon
paper_authors.type: user
paper_authors.user: Lune-Blue
paper_authors._id: 66fbbcffc247c5b3c24ee5d4
paper_authors.name: Yongho Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-01T13:24:47.537Z
paper_authors._id: 64c92f1a5381684d3eb1b0d1
paper_authors.avatarUrl: /avatars/35e500bcc486a274dbb9a420f4b17830.svg
paper_authors.fullname: Yongho Song
paper_authors.type: user
paper_authors.user: yonghosong
paper_authors._id: 66fbbcffc247c5b3c24ee5d5
paper_authors.name: Dongjin Kang
paper_authors._id: 66fbbcffc247c5b3c24ee5d6
paper_authors.name: Kai Tzu-iunn Ong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-01T13:25:06.059Z
paper_authors._id: 640ec2fd2f9c7b364d182735
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640ec2fd2f9c7b364d182735/G5WxQXYEOaVL39rAoOI3Y.jpeg
paper_authors.fullname: kai tzu-iunn ong
paper_authors.type: user
paper_authors.user: ktio
paper_authors._id: 66fbbcffc247c5b3c24ee5d7
paper_authors.name: Beong-woo Kwak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-01T13:25:13.937Z
paper_authors._id: 64b72a408ba7d6c922c73054
paper_authors.avatarUrl: /avatars/6d9797430bc36f05fb950b84aa6a9374.svg
paper_authors.fullname: Beong Woo Kwak
paper_authors.type: user
paper_authors.user: bwookwak
paper_authors._id: 66fbbcffc247c5b3c24ee5d8
paper_authors.name: Seonghyeon Bae
paper_authors._id: 66fbbcffc247c5b3c24ee5d9
paper_authors.name: Seung-won Hwang
paper_authors._id: 66fbbcffc247c5b3c24ee5da
paper_authors.name: Jinyoung Yeo
paper_title: Coffee-Gym: An Environment for Evaluating and Improving Natural Language
    Feedback on Erroneous Code
paper_summary: This paper presents Coffee-Gym, a comprehensive RL environment for training
  models that provide feedback on code editing. Coffee-Gym includes two major
  components: (1) Coffee, a dataset containing humans' code edit traces for
  coding questions and machine-written feedback for editing erroneous code; (2)
  CoffeeEval, a reward function that faithfully reflects the helpfulness of
  feedback by assessing the performance of the revised code in unit tests. With
  them, Coffee-Gym addresses the unavailability of high-quality datasets for
  training feedback models with RL, and provides more accurate rewards than the
  SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback
  models that outperform baselines in enhancing open-source code LLMs' code
  editing, making them comparable with closed-source LLMs. We make the dataset
  and the model checkpoint publicly available.
paper_ai_keywords: RL environment Coffee dataset code edit traces machine-written feedback CoffeeEval reward function performance assessment helpfulness of feedback tests high-quality datasets feedback models SOTA reward model GPT-4 code editing open-source code LLMs closed-source LLMs dataset model checkpoint