title: DreamReward: Text-to-3D Generation with Human Preference
summary: 3D content creation from text prompts has shown remarkable success recently.
  However, current text-to-3D methods often generate 3D results that do not align
  well with human preferences. In this paper, we present a comprehensive
  framework, coined DreamReward, to learn and improve text-to-3D models from
  human preference feedback. To begin with, we collect 25k expert comparisons
  based on a systematic annotation pipeline including rating and ranking. Then,
  we build Reward3D -- the first general-purpose text-to-3D human preference
  reward model to effectively encode human preferences. Building upon the 3D
  reward model, we finally perform theoretical analysis and present the Reward3D
  Feedback Learning (DreamFL), a direct tuning algorithm to optimize the
  multi-view diffusion models with a redefined scorer. Grounded by theoretical
  proof and extensive experiment comparisons, our DreamReward successfully
  generates high-fidelity and 3D consistent results with significant boosts in
  prompt alignment with human intention. Our results demonstrate the great
  potential for learning from human feedback to improve text-to-3D models.
paper_authors._id: 65fcf1dbc3f470c772d92e5a
paper_authors.name: Junliang Ye
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-25T09:44:07.989Z
paper_authors._id: 65a420cd90e65dc39a6abe9e
paper_authors.avatarUrl: /avatars/81ac5b749043e899f5017782409f9e28.svg
paper_authors.fullname: yejunliang
paper_authors.type: user
paper_authors.user: yejunliang23
paper_authors._id: 65fcf1dbc3f470c772d92e5b
paper_authors.name: Fangfu Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-22T08:48:56.419Z
paper_authors._id: 6505a02f9310ce8c400edc63
paper_authors.avatarUrl: /avatars/bbf781594fc8c812316711aa8e2797aa.svg
paper_authors.fullname: Fangfu Liu
paper_authors.type: user
paper_authors.user: Liuff23
paper_authors._id: 65fcf1dbc3f470c772d92e5c
paper_authors.name: Qixiu Li
paper_authors._id: 65fcf1dbc3f470c772d92e5d
paper_authors.name: Zhengyi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-22T08:49:32.174Z
paper_authors._id: 634e15aec1ce28f1de91c470
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/634e15aec1ce28f1de91c470/MdtJKgE_fbHydhqESr-DH.png
paper_authors.fullname: Zhengyi Wang
paper_authors.type: user
paper_authors.user: Zhengyi
paper_authors._id: 65fcf1dbc3f470c772d92e5e
paper_authors.name: Yikai Wang
paper_authors._id: 65fcf1dbc3f470c772d92e5f
paper_authors.name: Xinzhou Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-22T08:50:52.014Z
paper_authors._id: 6459c45b5a263a4c87b7c639
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/t3iK1wPBOP-h_kLBzZA_h.png
paper_authors.fullname: Xinzhou Wang
paper_authors.type: user
paper_authors.user: zz7379
paper_authors._id: 65fcf1dbc3f470c772d92e60
paper_authors.name: Yueqi Duan
paper_authors._id: 65fcf1dbc3f470c772d92e61
paper_authors.name: Jun Zhu
paper_title: DreamReward: Text-to-3D Generation with Human Preference
paper_summary: 3D content creation from text prompts has shown remarkable success recently.
  However, current text-to-3D methods often generate 3D results that do not align
  well with human preferences. In this paper, we present a comprehensive
  framework, coined DreamReward, to learn and improve text-to-3D models from
  human preference feedback. To begin with, we collect 25k expert comparisons
  based on a systematic annotation pipeline including rating and ranking. Then,
  we build Reward3D -- the first general-purpose text-to-3D human preference
  reward model to effectively encode human preferences. Building upon the 3D
  reward model, we finally perform theoretical analysis and present the Reward3D
  Feedback Learning (DreamFL), a direct tuning algorithm to optimize the
  multi-view diffusion models with a redefined scorer. Grounded by theoretical
  proof and extensive experiment comparisons, our DreamReward successfully
  generates high-fidelity and 3D consistent results with significant boosts in
  prompt alignment with human intention. Our results demonstrate the great
  potential for learning from human feedback to improve text-to-3D models.
paper_ai_keywords: DreamReward Reward3D Reward3D Feedback Learning (DreamFL) multi-view diffusion models hash tables