title: Rich Human Feedback for Text-to-Image Generation
summary: Recent Text-to-Image (T2I) generation models such as Stable Diffusion and
  Imagen have made significant progress in generating high-resolution images
  based on text descriptions. However, many generated images still suffer from
  issues such as artifacts/implausibility, misalignment with text descriptions,
  and low aesthetic quality. Inspired by the success of Reinforcement Learning
  with Human Feedback (RLHF) for large language models, prior works collected
  human-provided scores as feedback on generated images and trained a reward
  model to improve the T2I generation. In this paper, we enrich the feedback
  signal by (i) marking image regions that are implausible or misaligned with the
  text, and (ii) annotating which words in the text prompt are misrepresented or
  missing on the image. We collect such rich human feedback on 18K generated
  images and train a multimodal transformer to predict the rich feedback
  automatically. We show that the predicted rich human feedback can be leveraged
  to improve image generation, for example, by selecting high-quality training
  data to finetune and improve the generative models, or by creating masks with
  predicted heatmaps to inpaint the problematic regions. Notably, the
  improvements generalize to models (Muse) beyond those used to generate the
  images on which human feedback data were collected (Stable Diffusion variants).
paper_authors._id: 6581296adefc9c0d300959bb
paper_authors.name: Youwei Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:02:23.630Z
paper_authors._id: 64a475aa33ded98516c6a37b
paper_authors.avatarUrl: /avatars/3abf5f8c7f56cc8612ab590f1b75c1d5.svg
paper_authors.fullname: Youwei Liang
paper_authors.type: user
paper_authors.user: youweiliang
paper_authors._id: 6581296adefc9c0d300959bc
paper_authors.name: Junfeng He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:02:37.037Z
paper_authors._id: 65794433ab3436f5adcb5649
paper_authors.avatarUrl: /avatars/dde2ee2ba522bb1b657c9c782cac60b1.svg
paper_authors.fullname: hejunfeng
paper_authors.type: user
paper_authors.user: hejunfeng
paper_authors._id: 6581296adefc9c0d300959bd
paper_authors.name: Gang Li
paper_authors._id: 6581296adefc9c0d300959be
paper_authors.name: Peizhao Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:04:02.406Z
paper_authors._id: 652442ce3a3b8c4b40d1dfac
paper_authors.avatarUrl: /avatars/c6625570a7ac8c0233eb602652bbcbcb.svg
paper_authors.fullname: Peizhao Li
paper_authors.type: user
paper_authors.user: pzli
paper_authors._id: 6581296adefc9c0d300959bf
paper_authors.name: Arseniy Klimovskiy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-20T08:20:01.667Z
paper_authors._id: 6581f9b82db4c45ef3e6449b
paper_authors.avatarUrl: /avatars/12565c86e45f33feeef188955d329a86.svg
paper_authors.fullname: Arseniy Klimovskiy
paper_authors.type: user
paper_authors.user: aklimovsky
paper_authors._id: 6581296adefc9c0d300959c0
paper_authors.name: Nicholas Carolan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-19T19:21:43.761Z
paper_authors._id: 6581e9972db4c45ef3e3aede
paper_authors.avatarUrl: /avatars/244a0379a2c7bcfb6e70732971a50524.svg
paper_authors.fullname: Nicholas Carolan
paper_authors.type: user
paper_authors.user: ncarolan
paper_authors._id: 6581296adefc9c0d300959c1
paper_authors.name: Jiao Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:04:16.686Z
paper_authors._id: 642cd2f7ad221e8f41d2f05e
paper_authors.avatarUrl: /avatars/64ad33b67c10e3cb7b9b07744d108d8c.svg
paper_authors.fullname: Jiao Sun
paper_authors.type: user
paper_authors.user: sunjiao123sun
paper_authors._id: 6581296adefc9c0d300959c2
paper_authors.name: Jordi Pont-Tuset
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-02T09:24:34.582Z
paper_authors._id: 66335b4b07f864a2f94b403d
paper_authors.avatarUrl: /avatars/0a290942c4a282b48f870500cc6424b5.svg
paper_authors.fullname: Jordi Pont Tuset
paper_authors.type: user
paper_authors.user: jponttuset
paper_authors._id: 6581296adefc9c0d300959c3
paper_authors.name: Sarah Young
paper_authors._id: 6581296adefc9c0d300959c4
paper_authors.name: Feng Yang
paper_authors._id: 6581296adefc9c0d300959c5
paper_authors.name: Junjie Ke
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:05:49.856Z
paper_authors._id: 64736b148b7a55cfa91e000f
paper_authors.avatarUrl: /avatars/dcd2a666aba505575a79ffcc08a98c9d.svg
paper_authors.fullname: Ian Kerr
paper_authors.type: user
paper_authors.user: KeJunjie
paper_authors._id: 6581296adefc9c0d300959c6
paper_authors.name: Krishnamurthy Dj Dvijotham
paper_authors._id: 6581296adefc9c0d300959c7
paper_authors.name: Katie Collins
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T10:06:28.701Z
paper_authors._id: 64514cd06db3be7ee57b6f34
paper_authors.avatarUrl: /avatars/3276eeb344f57601e2a2199dd090935b.svg
paper_authors.fullname: Katie Collins
paper_authors.type: user
paper_authors.user: kmcollins
paper_authors._id: 6581296adefc9c0d300959c8
paper_authors.name: Yiwen Luo
paper_authors._id: 6581296adefc9c0d300959c9
paper_authors.name: Yang Li
paper_authors._id: 6581296adefc9c0d300959ca
paper_authors.name: Kai J Kohlhoff
paper_authors._id: 6581296adefc9c0d300959cb
paper_authors.name: Deepak Ramachandran
paper_authors._id: 6581296adefc9c0d300959cc
paper_authors.name: Vidhya Navalpakkam
paper_title: Rich Human Feedback for Text-to-Image Generation
paper_summary: Recent Text-to-Image (T2I) generation models such as Stable Diffusion and
  Imagen have made significant progress in generating high-resolution images
  based on text descriptions. However, many generated images still suffer from
  issues such as artifacts/implausibility, misalignment with text descriptions,
  and low aesthetic quality. Inspired by the success of Reinforcement Learning
  with Human Feedback (RLHF) for large language models, prior works collected
  human-provided scores as feedback on generated images and trained a reward
  model to improve the T2I generation. In this paper, we enrich the feedback
  signal by (i) marking image regions that are implausible or misaligned with the
  text, and (ii) annotating which words in the text prompt are misrepresented or
  missing on the image. We collect such rich human feedback on 18K generated
  images and train a multimodal transformer to predict the rich feedback
  automatically. We show that the predicted rich human feedback can be leveraged
  to improve image generation, for example, by selecting high-quality training
  data to finetune and improve the generative models, or by creating masks with
  predicted heatmaps to inpaint the problematic regions. Notably, the
  improvements generalize to models (Muse) beyond those used to generate the
  images on which human feedback data were collected (Stable Diffusion variants).
paper_ai_keywords: Text-to-Image (T2I) Stable Diffusion Imagen high-resolution images text descriptions Reinforcement Learning with Human Feedback (RLHF) large language models reward model image regions text prompt multimodal transformer rich human feedback training data finetune generative models masks heatmaps inpaint Muse