title: VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning
    for Image and Video Generation
summary: We present a general strategy to aligning visual generation models -- both
  image and video generation -- with human preference. To start with, we build
  VisionReward -- a fine-grained and multi-dimensional reward model. We decompose
  human preferences in images and videos into multiple dimensions, each
  represented by a series of judgment questions, linearly weighted and summed to
  an interpretable and accurate score. To address the challenges of video quality
  assessment, we systematically analyze various dynamic features of videos, which
  helps VisionReward surpass VideoScore by 17.2% and achieve top performance for
  video preference prediction. Based on VisionReward, we develop a
  multi-objective preference learning algorithm that effectively addresses the
  issue of confounding factors within preference data. Our approach significantly
  outperforms existing image and video scoring methods on both machine metrics
  and human evaluation. All code and datasets are provided at
  https://github.com/THUDM/VisionReward.
paper_authors._id: 677a6fb5deb62f25f4209c2a
paper_authors.name: Jiazheng Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-06T08:00:26.452Z
paper_authors._id: 62d7b131f6e8ba66107af761
paper_authors.avatarUrl: /avatars/f1c5df47aef69c824fd166722df8f670.svg
paper_authors.fullname: Jiazheng Xu
paper_authors.type: user
paper_authors.user: xujz0703
paper_authors._id: 677a6fb5deb62f25f4209c2b
paper_authors.name: Yu Huang
paper_authors._id: 677a6fb5deb62f25f4209c2c
paper_authors.name: Jiale Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:41:34.810Z
paper_authors._id: 627626d42d26ac639e56f565
paper_authors.avatarUrl: /avatars/805c5f909f52656345b8bde486c9fa8f.svg
paper_authors.fullname: Jiale Cheng
paper_authors.type: user
paper_authors.user: CCCCCC
paper_authors._id: 677a6fb5deb62f25f4209c2d
paper_authors.name: Yuanming Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:41:48.700Z
paper_authors._id: 659b6c50b0f43ed69fe09d56
paper_authors.avatarUrl: /avatars/8ea56c56263595a9f7555f2c2520641a.svg
paper_authors.fullname: 杨远明
paper_authors.type: user
paper_authors.user: yangyuanming
paper_authors._id: 677a6fb5deb62f25f4209c2e
paper_authors.name: Jiajun Xu
paper_authors._id: 677a6fb5deb62f25f4209c2f
paper_authors.name: Yuan Wang
paper_authors._id: 677a6fb5deb62f25f4209c30
paper_authors.name: Wenbo Duan
paper_authors._id: 677a6fb5deb62f25f4209c31
paper_authors.name: Shen Yang
paper_authors._id: 677a6fb5deb62f25f4209c32
paper_authors.name: Qunlin Jin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:42:43.541Z
paper_authors._id: 65007e0870b6b05c5ad94a62
paper_authors.avatarUrl: /avatars/c1ff713db5748db121738c601f8add85.svg
paper_authors.fullname: Jin Qunlin
paper_authors.type: user
paper_authors.user: kimtorch
paper_authors._id: 677a6fb5deb62f25f4209c33
paper_authors.name: Shurun Li
paper_authors._id: 677a6fb5deb62f25f4209c34
paper_authors.name: Jiayan Teng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:43:16.822Z
paper_authors._id: 65228733377bffdc59a10117
paper_authors.avatarUrl: /avatars/6eec07553658ab22f8058caa0bfbed49.svg
paper_authors.fullname: tengjiayan
paper_authors.type: user
paper_authors.user: tengjiayan
paper_authors._id: 677a6fb5deb62f25f4209c35
paper_authors.name: Zhuoyi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:43:40.186Z
paper_authors._id: 6466d1640ed2f7a8cba87503
paper_authors.avatarUrl: /avatars/652746e63dfeb5154ae7d34039d1a485.svg
paper_authors.fullname: Zhuoyi Yang
paper_authors.type: user
paper_authors.user: zyyangzy
paper_authors._id: 677a6fb5deb62f25f4209c36
paper_authors.name: Wendi Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:44:00.832Z
paper_authors._id: 650aaaf1634e02df56dfd231
paper_authors.avatarUrl: /avatars/4643ab23721d4ed6aeb1ebbc717adc43.svg
paper_authors.fullname: Wendi Zheng
paper_authors.type: user
paper_authors.user: zwd125
paper_authors._id: 677a6fb5deb62f25f4209c37
paper_authors.name: Xiao Liu
paper_authors._id: 677a6fb5deb62f25f4209c38
paper_authors.name: Ming Ding
paper_authors._id: 677a6fb5deb62f25f4209c39
paper_authors.name: Xiaohan Zhang
paper_authors._id: 677a6fb5deb62f25f4209c3a
paper_authors.name: Xiaotao Gu
paper_authors._id: 677a6fb5deb62f25f4209c3b
paper_authors.name: Shiyu Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:39:28.802Z
paper_authors._id: 6406db5cd684369027166986
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6406db5cd684369027166986/Zl-orrGcbY0RbfjfKszn1.jpeg
paper_authors.fullname: Shiyu Huang
paper_authors.type: user
paper_authors.user: ShiyuHuang
paper_authors._id: 677a6fb5deb62f25f4209c3c
paper_authors.name: Minlie Huang
paper_authors._id: 677a6fb5deb62f25f4209c3d
paper_authors.name: Jie Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:38:45.096Z
paper_authors._id: 640dff05474aa6f89556677e
paper_authors.avatarUrl: /avatars/1b4591c7322d649c797b3125148f1915.svg
paper_authors.fullname: Jie Tang
paper_authors.type: user
paper_authors.user: jerytang
paper_authors._id: 677a6fb5deb62f25f4209c3e
paper_authors.name: Yuxiao Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T12:38:37.430Z
paper_authors._id: 640e73bdfdeaae1390857b62
paper_authors.avatarUrl: /avatars/cd6779e30f716002a7838ed93d5c0754.svg
paper_authors.fullname: Yuxiao Dong
paper_authors.type: user
paper_authors.user: yuxiaod
paper_title: VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning
    for Image and Video Generation
paper_summary: We present a general strategy to aligning visual generation models -- both
  image and video generation -- with human preference. To start with, we build
  VisionReward -- a fine-grained and multi-dimensional reward model. We decompose
  human preferences in images and videos into multiple dimensions, each
  represented by a series of judgment questions, linearly weighted and summed to
  an interpretable and accurate score. To address the challenges of video quality
  assessment, we systematically analyze various dynamic features of videos, which
  helps VisionReward surpass VideoScore by 17.2% and achieve top performance for
  video preference prediction. Based on VisionReward, we develop a
  multi-objective preference learning algorithm that effectively addresses the
  issue of confounding factors within preference data. Our approach significantly
  outperforms existing image and video scoring methods on both machine metrics
  and human evaluation. All code and datasets are provided at
  https://github.com/THUDM/VisionReward.
paper_ai_keywords: VisionReward fine-grained multi-dimensional reward model judgment questions VideoScore dynamic features multi-objective preference learning algorithm confounding factors