title: Preference Tuning with Human Feedback on Language, Speech, and Vision
    Tasks: A Survey
summary: Preference tuning is a crucial process for aligning deep generative models
  with human preferences. This survey offers a thorough overview of recent
  advancements in preference tuning and the integration of human feedback. The
  paper is organized into three main sections: 1) introduction and preliminaries:
  an introduction to reinforcement learning frameworks, preference tuning tasks,
  models, and datasets across various modalities: language, speech, and vision,
  as well as different policy approaches, 2) in-depth examination of each
  preference tuning approach: a detailed analysis of the methods used in
  preference tuning, and 3) applications, discussion, and future directions: an
  exploration of the applications of preference tuning in downstream tasks,
  including evaluation methods for different modalities, and an outlook on future
  research directions. Our objective is to present the latest methodologies in
  preference tuning and model alignment, enhancing the understanding of this
  field for researchers and practitioners. We hope to encourage further
  engagement and innovation in this area.
paper_authors._id: 66eb9444d346952f358d6d5f
paper_authors.name: Genta Indra Winata
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-19T08:37:33.639Z
paper_authors._id: 5f5c4b20e56d546cd6233098
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1637813888895-5f5c4b20e56d546cd6233098.jpeg
paper_authors.fullname: Genta Indra Winata
paper_authors.type: user
paper_authors.user: gentaiscool
paper_authors._id: 66eb9444d346952f358d6d60
paper_authors.name: Hanyang Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-19T08:37:39.486Z
paper_authors._id: 65543320520d9bb9d9d3d47d
paper_authors.avatarUrl: /avatars/89808c80248af4e234bdd42abea94d91.svg
paper_authors.fullname: Hanyang Zhao
paper_authors.type: user
paper_authors.user: BraceZHY
paper_authors._id: 66eb9444d346952f358d6d61
paper_authors.name: Anirban Das
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-19T08:38:15.906Z
paper_authors._id: 64b70e5afdb702b3d85a6005
paper_authors.avatarUrl: /avatars/e3d87d05895583aa0eec1d197a56c986.svg
paper_authors.fullname: Anirban Das
paper_authors.type: user
paper_authors.user: anirbandas
paper_authors._id: 66eb9444d346952f358d6d62
paper_authors.name: Wenpin Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-19T18:38:55.582Z
paper_authors._id: 66ec3fbdce3f72f90d54c1e4
paper_authors.avatarUrl: /avatars/045eacff25b9efd9daa6e19ec0f134a1.svg
paper_authors.fullname: Wenpin Tang
paper_authors.type: user
paper_authors.user: wt2319
paper_authors._id: 66eb9444d346952f358d6d63
paper_authors.name: David D. Yao
paper_authors._id: 66eb9444d346952f358d6d64
paper_authors.name: Shi-Xiong Zhang
paper_authors._id: 66eb9444d346952f358d6d65
paper_authors.name: Sambit Sahu
paper_title: Preference Tuning with Human Feedback on Language, Speech, and Vision
    Tasks: A Survey
paper_summary: Preference tuning is a crucial process for aligning deep generative models
  with human preferences. This survey offers a thorough overview of recent
  advancements in preference tuning and the integration of human feedback. The
  paper is organized into three main sections: 1) introduction and preliminaries:
  an introduction to reinforcement learning frameworks, preference tuning tasks,
  models, and datasets across various modalities: language, speech, and vision,
  as well as different policy approaches, 2) in-depth examination of each
  preference tuning approach: a detailed analysis of the methods used in
  preference tuning, and 3) applications, discussion, and future directions: an
  exploration of the applications of preference tuning in downstream tasks,
  including evaluation methods for different modalities, and an outlook on future
  research directions. Our objective is to present the latest methodologies in
  preference tuning and model alignment, enhancing the understanding of this
  field for researchers and practitioners. We hope to encourage further
  engagement and innovation in this area.
paper_ai_keywords: preference tuning deep generative models human preferences reinforcement learning frameworks preference tuning tasks policy approaches language speech vision applications downstream tasks evaluation methods model alignment