title: Prometheus: Inducing Fine-grained Evaluation Capability in Language
    Models
summary: Recently, using a powerful proprietary Large Language Model (LLM) (e.g.,
  GPT-4) as an evaluator for long-form responses has become the de facto
  standard. However, for practitioners with large-scale evaluation tasks and
  custom criteria in consideration (e.g., child-readability), using proprietary
  LLMs as an evaluator is unreliable due to the closed-source nature,
  uncontrolled versioning, and prohibitive costs. In this work, we propose
  Prometheus, a fully open-source LLM that is on par with GPT-4's evaluation
  capabilities when the appropriate reference materials (reference answer, score
  rubric) are accompanied. We first construct the Feedback Collection, a new
  dataset that consists of 1K fine-grained score rubrics, 20K instructions, and
  100K responses and language feedback generated by GPT-4. Using the Feedback
  Collection, we train Prometheus, a 13B evaluator LLM that can assess any given
  long-form text based on customized score rubric provided by the user.
  Experimental results show that Prometheus scores a Pearson correlation of 0.897
  with human evaluators when evaluating with 45 customized score rubrics, which
  is on par with GPT-4 (0.882), and greatly outperforms ChatGPT (0.392).
  Furthermore, measuring correlation with GPT-4 with 1222 customized score
  rubrics across four benchmarks (MT Bench, Vicuna Bench, Feedback Bench, Flask
  Eval) shows similar trends, bolstering Prometheus's capability as an evaluator
  LLM. Lastly, Prometheus achieves the highest accuracy on two human preference
  benchmarks (HHH Alignment & MT Bench Human Judgment) compared to open-sourced
  reward models explicitly trained on human preference datasets, highlighting its
  potential as an universal reward model. We open-source our code, dataset, and
  model at https://github.com/kaistAI/Prometheus.
paper_authors._id: 65289af6aeb01f59a14fbd72
paper_authors.name: Seungone Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-13T07:52:08.713Z
paper_authors._id: 6469949654873f0043b09c22
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6469949654873f0043b09c22/Lk7IJAR16Wa_sGJ2g81AQ.jpeg
paper_authors.fullname: Seungone Kim
paper_authors.type: user
paper_authors.user: seungone
paper_authors._id: 65289af6aeb01f59a14fbd73
paper_authors.name: Jamin Shin
paper_authors._id: 65289af6aeb01f59a14fbd74
paper_authors.name: Yejin Cho
paper_authors._id: 65289af6aeb01f59a14fbd75
paper_authors.name: Joel Jang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T08:39:42.278Z
paper_authors._id: 613e1a9267835521a6816b04
paper_authors.avatarUrl: /avatars/49edaa425bbce04dff92bbfb12a6b41c.svg
paper_authors.fullname: Joel Jang
paper_authors.type: user
paper_authors.user: wkddydpf
paper_authors._id: 65289af6aeb01f59a14fbd76
paper_authors.name: Shayne Longpre
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T08:39:49.890Z
paper_authors._id: 61f4283a81c4d30f58140242
paper_authors.avatarUrl: /avatars/a1cf1ef1fd442c36ed65c68e51919fed.svg
paper_authors.fullname: Shayne Longpre
paper_authors.type: user
paper_authors.user: Shayne
paper_authors._id: 65289af6aeb01f59a14fbd77
paper_authors.name: Hwaran Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-13T14:42:58.184Z
paper_authors._id: 65294f43375b3a8bc1351acd
paper_authors.avatarUrl: /avatars/0105b9b152bc897dd36d0d9b8a590ddf.svg
paper_authors.fullname: Hwaran Lee
paper_authors.type: user
paper_authors.user: hwaranlee
paper_authors._id: 65289af6aeb01f59a14fbd78
paper_authors.name: Sangdoo Yun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T08:40:05.477Z
paper_authors._id: 648ac415718bc0670a5a5f56
paper_authors.avatarUrl: /avatars/27189e289b808ef01689ff2abb7a56bf.svg
paper_authors.fullname: Sangdoo Yun
paper_authors.type: user
paper_authors.user: oodgnas
paper_authors._id: 65289af6aeb01f59a14fbd79
paper_authors.name: Seongjin Shin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T08:40:18.610Z
paper_authors._id: 631b8731824f2502e353ccdb
paper_authors.avatarUrl: /avatars/7b053800e3cf35b61d4af15180618f23.svg
paper_authors.fullname: SHIN SEONGJIN
paper_authors.type: user
paper_authors.user: optimalc
paper_authors._id: 65289af6aeb01f59a14fbd7a
paper_authors.name: Sungdong Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T08:40:27.204Z
paper_authors._id: 6084ebe956f7623a0b849153
paper_authors.avatarUrl: /avatars/c56fb056800c145e5a8234e27b6b5a31.svg
paper_authors.fullname: Sungdong kim
paper_authors.type: user
paper_authors.user: dsksd
paper_authors._id: 65289af6aeb01f59a14fbd7b
paper_authors.name: James Thorne
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T08:41:03.493Z
paper_authors._id: 63b54db5889aa6707f08f2d1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63b54db5889aa6707f08f2d1/zUnyx-8Zabxr54poZtUo4.jpeg
paper_authors.fullname: James Thorne
paper_authors.type: user
paper_authors.user: j6mes
paper_authors._id: 65289af6aeb01f59a14fbd7c
paper_authors.name: Minjoon Seo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T08:41:11.137Z
paper_authors._id: 621f05ba970615ad5861ceb1
paper_authors.avatarUrl: /avatars/7e1902aa71369a524afda9b0a9e88e22.svg
paper_authors.fullname: Minjoon Seo
paper_authors.type: user
paper_authors.user: minjoon
paper_title: Prometheus: Inducing Fine-grained Evaluation Capability in Language
    Models
paper_summary: Recently, using a powerful proprietary Large Language Model (LLM) (e.g.,
  GPT-4) as an evaluator for long-form responses has become the de facto
  standard. However, for practitioners with large-scale evaluation tasks and
  custom criteria in consideration (e.g., child-readability), using proprietary
  LLMs as an evaluator is unreliable due to the closed-source nature,
  uncontrolled versioning, and prohibitive costs. In this work, we propose
  Prometheus, a fully open-source LLM that is on par with GPT-4's evaluation
  capabilities when the appropriate reference materials (reference answer, score
  rubric) are accompanied. We first construct the Feedback Collection, a new
  dataset that consists of 1K fine-grained score rubrics, 20K instructions, and
  100K responses and language feedback generated by GPT-4. Using the Feedback
  Collection, we train Prometheus, a 13B evaluator LLM that can assess any given
  long-form text based on customized score rubric provided by the user.
  Experimental results show that Prometheus scores a Pearson correlation of 0.897
  with human evaluators when evaluating with 45 customized score rubrics, which
  is on par with GPT-4 (0.882), and greatly outperforms ChatGPT (0.392).
  Furthermore, measuring correlation with GPT-4 with 1222 customized score
  rubrics across four benchmarks (MT Bench, Vicuna Bench, Feedback Bench, Flask
  Eval) shows similar trends, bolstering Prometheus's capability as an evaluator
  LLM. Lastly, Prometheus achieves the highest accuracy on two human preference
  benchmarks (HHH Alignment & MT Bench Human Judgment) compared to open-sourced
  reward models explicitly trained on human preference datasets, highlighting its
  potential as an universal reward model. We open-source our code, dataset, and
  model at https://github.com/kaistAI/Prometheus.
paper_ai_keywords: Large Language Model (LLM) GPT-4 Prometheus Feedback Collection fine-grained score rubrics instructions responses language feedback Pearson correlation human evaluators customized score rubrics ChatGPT MT Bench Vicuna Bench Feedback Bench Flask Eval human preference benchmarks HHH Alignment MT Bench Human Judgment universal reward model