title: Shepherd: A Critic for Language Model Generation
summary: As large language models improve, there is increasing interest in techniques
  that leverage these models' capabilities to refine their own outputs. In this
  work, we introduce Shepherd, a language model specifically tuned to critique
  responses and suggest refinements, extending beyond the capabilities of an
  untuned model to identify diverse errors and provide suggestions to remedy
  them. At the core of our approach is a high quality feedback dataset, which we
  curate from community feedback and human annotations. Even though Shepherd is
  small (7B parameters), its critiques are either equivalent or preferred to
  those from established models including ChatGPT. Using GPT-4 for evaluation,
  Shepherd reaches an average win-rate of 53-87% compared to competitive
  alternatives. In human evaluation, Shepherd strictly outperforms other models
  and on average closely ties with ChatGPT.
paper_authors._id: 64d4423823996fa86776ef1c
paper_authors.name: Tianlu Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-10T12:56:16.108Z
paper_authors._id: 63f6a0e6b4c9a104f4b95d86
paper_authors.avatarUrl: /avatars/660c3e44bea976f3e0560bf95466b03c.svg
paper_authors.fullname: Tianlu Wang
paper_authors.type: user
paper_authors.user: Tianlu
paper_authors._id: 64d4423823996fa86776ef1d
paper_authors.name: Ping Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-11T02:11:32.520Z
paper_authors._id: 646c004e31968a60a022590e
paper_authors.avatarUrl: /avatars/349e2ed4feef1be050fd9903dccec1e1.svg
paper_authors.fullname: Ping Yu
paper_authors.type: user
paper_authors.user: pingyu
paper_authors._id: 64d4423823996fa86776ef1e
paper_authors.name: Xiaoqing Ellen Tan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-11T02:11:36.576Z
paper_authors._id: 6345ab8ffe134dfd7a0d1059
paper_authors.avatarUrl: /avatars/159a364ac4f8a2ed001346d4cbb7f2e3.svg
paper_authors.fullname: Ellen Tan
paper_authors.type: user
paper_authors.user: ellenxtan
paper_authors._id: 64d4423823996fa86776ef1f
paper_authors.name: Sean O'Brien
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-10T13:34:00.483Z
paper_authors._id: 63f5451571a5d395c7212c94
paper_authors.avatarUrl: /avatars/05e199ab2b5e2fe5403fc63b8825daf7.svg
paper_authors.fullname: Sean O'Brien
paper_authors.type: user
paper_authors.user: Sean-OB
paper_authors._id: 64d4423823996fa86776ef20
paper_authors.name: Ramakanth Pasunuru
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-10T13:34:17.793Z
paper_authors._id: 63b4be2334df1f9a90a1983c
paper_authors.avatarUrl: /avatars/9cdfa3492c87aabdacab684b80628f14.svg
paper_authors.fullname: Ramakanth Pasunuru
paper_authors.type: user
paper_authors.user: rpasunuru
paper_authors._id: 64d4423823996fa86776ef21
paper_authors.name: Jane Dwivedi-Yu
paper_authors._id: 64d4423823996fa86776ef22
paper_authors.name: Olga Golovneva
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-10T13:34:45.589Z
paper_authors._id: 6318b1e06e6bd039440871d6
paper_authors.avatarUrl: /avatars/2d9f96eb4092a9d1734dcc09303b259a.svg
paper_authors.fullname: Olga Golovneva
paper_authors.type: user
paper_authors.user: Golovneva
paper_authors._id: 64d4423823996fa86776ef23
paper_authors.name: Luke Zettlemoyer
paper_authors._id: 64d4423823996fa86776ef24
paper_authors.name: Maryam Fazel-Zarandi
paper_authors._id: 64d4423823996fa86776ef25
paper_authors.name: Asli Celikyilmaz
paper_title: Shepherd: A Critic for Language Model Generation
paper_summary: As large language models improve, there is increasing interest in techniques
  that leverage these models' capabilities to refine their own outputs. In this
  work, we introduce Shepherd, a language model specifically tuned to critique
  responses and suggest refinements, extending beyond the capabilities of an
  untuned model to identify diverse errors and provide suggestions to remedy
  them. At the core of our approach is a high quality feedback dataset, which we
  curate from community feedback and human annotations. Even though Shepherd is
  small (7B parameters), its critiques are either equivalent or preferred to
  those from established models including ChatGPT. Using GPT-4 for evaluation,
  Shepherd reaches an average win-rate of 53-87% compared to competitive
  alternatives. In human evaluation, Shepherd strictly outperforms other models
  and on average closely ties with ChatGPT.
paper_ai_summary: Shepherd, a small language model tuned for critique, outperforms or ties with larger models like ChatGPT in refining language model outputs using a high-quality feedback dataset.
paper_ai_keywords: language model critique feedback dataset GPT-4 human evaluation ChatGPT win-rate