System-Level Natural Language Feedback
Natural language (NL) feedback contains rich information about the user
experience. Existing studies focus on an instance-level approach, where
feedback is used to refine specific examples, disregarding its system-wide
application. This paper proposes a general framework for unlocking the
system-level use of NL feedback. We show how to use feedback to formalize
system-level design decisions in a human-in-the-loop-process -- in order to
produce better models. In particular this is done through: (i) metric design
for tasks; and (ii) language model prompt design for refining model responses.
We conduct two case studies of this approach for improving search query
generation and dialog response generation, demonstrating the effectiveness of
the use of system-level feedback. We show the combination of system-level
feedback and instance-level feedback brings further gains, and that human
written instance-level feedback results in more grounded refinements than
GPT-3.5 written ones, underlying the importance of human feedback for building
systems.
649920227ac1bf03e21dfb98
Weizhe Yuan
claimed_verified
2023-06-27T07:27:42.308Z
6172aa78c8e66e2aa84c06b8
https://cdn-avatars.huggingface.co/v1/production/uploads/1634904686602-noauth.png
Weizhe Yuan
user
weizhey
649920227ac1bf03e21dfb99
Kyunghyun Cho
claimed_verified
2023-06-28T13:33:07.111Z
649c1b8485b05ee02723334a
/avatars/29080f550fdb471c4d2daace86aff3b0.svg
Kyunghyun Cho
user
kyunghyuncho
649920227ac1bf03e21dfb9a
Jason Weston
claimed_verified
2023-06-27T07:27:38.250Z
62f023a36a027498eaa2f9cc
/avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
Jason Weston
user
spermwhale
System-Level Natural Language Feedback
Natural language (NL) feedback contains rich information about the user
experience. Existing studies focus on an instance-level approach, where
feedback is used to refine specific examples, disregarding its system-wide
application. This paper proposes a general framework for unlocking the
system-level use of NL feedback. We show how to use feedback to formalize
system-level design decisions in a human-in-the-loop-process -- in order to
produce better models. In particular this is done through: (i) metric design
for tasks; and (ii) language model prompt design for refining model responses.
We conduct two case studies of this approach for improving search query
generation and dialog response generation, demonstrating the effectiveness of
the use of system-level feedback. We show the combination of system-level
feedback and instance-level feedback brings further gains, and that human
written instance-level feedback results in more grounded refinements than
GPT-3.5 written ones, underlying the importance of human feedback for building
systems.
A framework is proposed to utilize system-level natural language feedback to improve model design and performance through metric design and prompt refinement, with case studies showing its effectiveness in search and dialog generation.
human-in-the-loop metric design language model prompt design system-level feedback instance-level feedback grounded refinements