title: Evaluating the role of `Constitutions' for learning from AI feedback
summary: The growing capabilities of large language models (LLMs) have led to their
  use as substitutes for human feedback for training and assessing other LLMs.
  These methods often rely on `constitutions', written guidelines which a critic
  model uses to provide feedback and improve generations. We investigate how the
  choice of constitution affects feedback quality by using four different
  constitutions to improve patient-centered communication in medical interviews.
  In pairwise comparisons conducted by 215 human raters, we found that detailed
  constitutions led to better results regarding emotive qualities. However, none
  of the constitutions outperformed the baseline in learning more
  practically-oriented skills related to information gathering and provision. Our
  findings indicate that while detailed constitutions should be prioritised,
  there are possible limitations to the effectiveness of AI feedback as a reward
  signal in certain areas.
paper_authors._id: 673c87d3dc27d069a3c808f7
paper_authors.name: Saskia Redgate
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T16:40:47.383Z
paper_authors._id: 65f1e0e65c4e61a391f087de
paper_authors.avatarUrl: /avatars/02248cb36255cd18583ceead0839fd6e.svg
paper_authors.fullname: Saskia Redgate
paper_authors.type: user
paper_authors.user: saskiar
paper_authors._id: 673c87d3dc27d069a3c808f8
paper_authors.name: Andrew M. Bean
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T16:40:55.178Z
paper_authors._id: 659bec4728676374f33ef921
paper_authors.avatarUrl: /avatars/217ae547d6460e65c6d2a23012741830.svg
paper_authors.fullname: Andrew Bean
paper_authors.type: user
paper_authors.user: ambean
paper_authors._id: 673c87d3dc27d069a3c808f9
paper_authors.name: Adam Mahdi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-20T10:09:01.008Z
paper_authors._id: 673cf9c93af47d1d2b6584c9
paper_authors.avatarUrl: /avatars/0792921a8adaa1493255abb0b5aaa4f1.svg
paper_authors.fullname: Adam Mahdi
paper_authors.type: user
paper_authors.user: ammaox
paper_title: Evaluating the role of `Constitutions' for learning from AI feedback
paper_summary: The growing capabilities of large language models (LLMs) have led to their
  use as substitutes for human feedback for training and assessing other LLMs.
  These methods often rely on `constitutions', written guidelines which a critic
  model uses to provide feedback and improve generations. We investigate how the
  choice of constitution affects feedback quality by using four different
  constitutions to improve patient-centered communication in medical interviews.
  In pairwise comparisons conducted by 215 human raters, we found that detailed
  constitutions led to better results regarding emotive qualities. However, none
  of the constitutions outperformed the baseline in learning more
  practically-oriented skills related to information gathering and provision. Our
  findings indicate that while detailed constitutions should be prioritised,
  there are possible limitations to the effectiveness of AI feedback as a reward
  signal in certain areas.
paper_ai_summary: Investigation into the impact of different constitutions on AI-generated feedback shows detailed constitutions improve emotive qualities in medical interviews but not practical skills.
paper_ai_keywords: large language models constitutions critic model patient-centered communication medical interviews pairwise comparisons emotive qualities information gathering provision AI feedback reward signal