title: System-Level Natural Language Feedback
summary: Natural language (NL) feedback contains rich information about the user
  experience. Existing studies focus on an instance-level approach, where
  feedback is used to refine specific examples, disregarding its system-wide
  application. This paper proposes a general framework for unlocking the
  system-level use of NL feedback. We show how to use feedback to formalize
  system-level design decisions in a human-in-the-loop-process -- in order to
  produce better models. In particular this is done through: (i) metric design
  for tasks; and (ii) language model prompt design for refining model responses.
  We conduct two case studies of this approach for improving search query
  generation and dialog response generation, demonstrating the effectiveness of
  the use of system-level feedback. We show the combination of system-level
  feedback and instance-level feedback brings further gains, and that human
  written instance-level feedback results in more grounded refinements than
  GPT-3.5 written ones, underlying the importance of human feedback for building
  systems.
paper_authors._id: 649920227ac1bf03e21dfb98
paper_authors.name: Weizhe Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-27T07:27:42.308Z
paper_authors._id: 6172aa78c8e66e2aa84c06b8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634904686602-noauth.png
paper_authors.fullname: Weizhe Yuan
paper_authors.type: user
paper_authors.user: weizhey
paper_authors._id: 649920227ac1bf03e21dfb99
paper_authors.name: Kyunghyun Cho
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-28T13:33:07.111Z
paper_authors._id: 649c1b8485b05ee02723334a
paper_authors.avatarUrl: /avatars/29080f550fdb471c4d2daace86aff3b0.svg
paper_authors.fullname: Kyunghyun Cho
paper_authors.type: user
paper_authors.user: kyunghyuncho
paper_authors._id: 649920227ac1bf03e21dfb9a
paper_authors.name: Jason Weston
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-27T07:27:38.250Z
paper_authors._id: 62f023a36a027498eaa2f9cc
paper_authors.avatarUrl: /avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
paper_authors.fullname: Jason Weston
paper_authors.type: user
paper_authors.user: spermwhale
paper_title: System-Level Natural Language Feedback
paper_summary: Natural language (NL) feedback contains rich information about the user
  experience. Existing studies focus on an instance-level approach, where
  feedback is used to refine specific examples, disregarding its system-wide
  application. This paper proposes a general framework for unlocking the
  system-level use of NL feedback. We show how to use feedback to formalize
  system-level design decisions in a human-in-the-loop-process -- in order to
  produce better models. In particular this is done through: (i) metric design
  for tasks; and (ii) language model prompt design for refining model responses.
  We conduct two case studies of this approach for improving search query
  generation and dialog response generation, demonstrating the effectiveness of
  the use of system-level feedback. We show the combination of system-level
  feedback and instance-level feedback brings further gains, and that human
  written instance-level feedback results in more grounded refinements than
  GPT-3.5 written ones, underlying the importance of human feedback for building
  systems.