Preference Tuning with Human Feedback on Language, Speech, and Vision
  Tasks: A Survey
Preference tuning is a crucial process for aligning deep generative models
with human preferences. This survey offers a thorough overview of recent
advancements in preference tuning and the integration of human feedback. The
paper is organized into three main sections: 1) introduction and preliminaries:
an introduction to reinforcement learning frameworks, preference tuning tasks,
models, and datasets across various modalities: language, speech, and vision,
as well as different policy approaches, 2) in-depth examination of each
preference tuning approach: a detailed analysis of the methods used in
preference tuning, and 3) applications, discussion, and future directions: an
exploration of the applications of preference tuning in downstream tasks,
including evaluation methods for different modalities, and an outlook on future
research directions. Our objective is to present the latest methodologies in
preference tuning and model alignment, enhancing the understanding of this
field for researchers and practitioners. We hope to encourage further
engagement and innovation in this area.
66eb9444d346952f358d6d5f
Genta Indra Winata
admin_assigned
2024-09-19T08:37:33.639Z
5f5c4b20e56d546cd6233098
https://cdn-avatars.huggingface.co/v1/production/uploads/1637813888895-5f5c4b20e56d546cd6233098.jpeg
Genta Indra Winata
user
gentaiscool
66eb9444d346952f358d6d60
Hanyang Zhao
admin_assigned
2024-09-19T08:37:39.486Z
65543320520d9bb9d9d3d47d
/avatars/89808c80248af4e234bdd42abea94d91.svg
Hanyang Zhao
user
BraceZHY
66eb9444d346952f358d6d61
Anirban Das
admin_assigned
2024-09-19T08:38:15.906Z
64b70e5afdb702b3d85a6005
/avatars/e3d87d05895583aa0eec1d197a56c986.svg
Anirban Das
user
anirbandas
66eb9444d346952f358d6d62
Wenpin Tang
claimed_verified
2024-09-19T18:38:55.582Z
66ec3fbdce3f72f90d54c1e4
/avatars/045eacff25b9efd9daa6e19ec0f134a1.svg
Wenpin Tang
user
wt2319
66eb9444d346952f358d6d63
David D. Yao
66eb9444d346952f358d6d64
Shi-Xiong Zhang
66eb9444d346952f358d6d65
Sambit Sahu
Preference Tuning with Human Feedback on Language, Speech, and Vision
  Tasks: A Survey
Preference tuning is a crucial process for aligning deep generative models
with human preferences. This survey offers a thorough overview of recent
advancements in preference tuning and the integration of human feedback. The
paper is organized into three main sections: 1) introduction and preliminaries:
an introduction to reinforcement learning frameworks, preference tuning tasks,
models, and datasets across various modalities: language, speech, and vision,
as well as different policy approaches, 2) in-depth examination of each
preference tuning approach: a detailed analysis of the methods used in
preference tuning, and 3) applications, discussion, and future directions: an
exploration of the applications of preference tuning in downstream tasks,
including evaluation methods for different modalities, and an outlook on future
research directions. Our objective is to present the latest methodologies in
preference tuning and model alignment, enhancing the understanding of this
field for researchers and practitioners. We hope to encourage further
engagement and innovation in this area.
Preference tuning in deep generative models integrates human feedback across modalities using reinforcement learning to align model outputs with human preferences, with a focus on methodologies and future directions.
preference tuning deep generative models human preferences reinforcement learning preference tuning tasks policy approaches language speech vision evaluation methods model alignment