title: Emu: Enhancing Image Generation Models Using Photogenic Needles in a
    Haystack
summary: Training text-to-image models with web scale image-text pairs enables the
  generation of a wide range of visual concepts from text. However, these
  pre-trained models often face challenges when it comes to generating highly
  aesthetic images. This creates the need for aesthetic alignment post
  pre-training. In this paper, we propose quality-tuning to effectively guide a
  pre-trained model to exclusively generate highly visually appealing images,
  while maintaining generality across visual concepts. Our key insight is that
  supervised fine-tuning with a set of surprisingly small but extremely visually
  appealing images can significantly improve the generation quality. We pre-train
  a latent diffusion model on 1.1 billion image-text pairs and fine-tune it
  with only a few thousand carefully selected high-quality images. The resulting
  model, Emu, achieves a win rate of 82.9% compared with its pre-trained only
  counterpart. Compared to the state-of-the-art SDXLv1.0, Emu is preferred
  68.4% and 71.3% of the time on visual appeal on the standard PartiPrompts
  and our Open User Input benchmark based on the real-world usage of
  text-to-image models. In addition, we show that quality-tuning is a generic
  approach that is also effective for other architectures, including pixel
  diffusion and masked generative transformer models.
paper_authors._id: 651503184b4f53b8268ac63b
paper_authors.name: Xiaoliang Dai
paper_authors._id: 651503184b4f53b8268ac63c
paper_authors.name: Ji Hou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:41:36.905Z
paper_authors._id: 63084636f0dc38fb47c2dc6b
paper_authors.avatarUrl: /avatars/04edff10a048b5d93981e6f5af3ecf40.svg
paper_authors.fullname: Ji Hou
paper_authors.type: user
paper_authors.user: jhou90
paper_authors._id: 651503184b4f53b8268ac63d
paper_authors.name: Chih-Yao Ma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-28T19:03:28.426Z
paper_authors._id: 6515bf00c2aedaa7e692a1ec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6515bf00c2aedaa7e692a1ec/PYpS1QwTPdQYW5_BrvwzP.png
paper_authors.fullname: Chih-Yao Ma
paper_authors.type: user
paper_authors.user: chihyaoma
paper_authors._id: 651503184b4f53b8268ac63e
paper_authors.name: Sam Tsai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T15:31:28.279Z
paper_authors._id: 67127b753a058517519fabe8
paper_authors.avatarUrl: /avatars/6fabb1634efb8ad8a8a2ae96fd6cf1c6.svg
paper_authors.fullname: Sam Tsai
paper_authors.type: user
paper_authors.user: sstsai-adl
paper_authors._id: 651503184b4f53b8268ac63f
paper_authors.name: Jialiang Wang
paper_authors._id: 651503184b4f53b8268ac640
paper_authors.name: Rui Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-28T19:03:31.458Z
paper_authors._id: 6515bcb7f464644a73b3ed8e
paper_authors.avatarUrl: /avatars/17872687e0117ffbdc2dcf0ddf273a60.svg
paper_authors.fullname: Rui Wang
paper_authors.type: user
paper_authors.user: theruiwang
paper_authors._id: 651503184b4f53b8268ac641
paper_authors.name: Peizhao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:43:07.975Z
paper_authors._id: 6433a635a5aed21dd110381e
paper_authors.avatarUrl: /avatars/b1fa4fe1c8bd7989875233820fa425af.svg
paper_authors.fullname: Peizhao Zhang
paper_authors.type: user
paper_authors.user: stzpz
paper_authors._id: 651503184b4f53b8268ac642
paper_authors.name: Simon Vandenhende
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:43:15.471Z
paper_authors._id: 62d9d1a4cf646c73e103524b
paper_authors.avatarUrl: /avatars/f29da9044d0f98241b5114b583ce3453.svg
paper_authors.fullname: Simon Vandenhende
paper_authors.type: user
paper_authors.user: svandenh
paper_authors._id: 651503184b4f53b8268ac643
paper_authors.name: Xiaofang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:43:23.770Z
paper_authors._id: 6334c1b3762d2d0e9eb151ea
paper_authors.avatarUrl: /avatars/45b3eb63f0515ab417aacd79aa9f8a41.svg
paper_authors.fullname: Xiaofang Wang
paper_authors.type: user
paper_authors.user: minione
paper_authors._id: 651503184b4f53b8268ac644
paper_authors.name: Abhimanyu Dubey
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:43:30.965Z
paper_authors._id: 6376f5a98272ecd8ddacacf8
paper_authors.avatarUrl: /avatars/c57d5de9a90ec12b7f617327b1ac27fd.svg
paper_authors.fullname: Abhimanyu Dubey
paper_authors.type: user
paper_authors.user: dubeya
paper_authors._id: 651503184b4f53b8268ac645
paper_authors.name: Matthew Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:43:48.250Z
paper_authors._id: 64043b47dbfbea2a0544e39d
paper_authors.avatarUrl: /avatars/fc985078067c06a8ac3f70b1cade158a.svg
paper_authors.fullname: Matthew Yu
paper_authors.type: user
paper_authors.user: mattcyu
paper_authors._id: 651503184b4f53b8268ac646
paper_authors.name: Abhishek Kadian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:44:16.919Z
paper_authors._id: 62c88a674fd4a24e1523f478
paper_authors.avatarUrl: /avatars/5a21438eca713bd97f5a2c8c5cbd980a.svg
paper_authors.fullname: Abhishek Kadian
paper_authors.type: user
paper_authors.user: abhiskk
paper_authors._id: 651503184b4f53b8268ac647
paper_authors.name: Filip Radenovic
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-29T07:04:28.704Z
paper_authors._id: 6515dccbc979eb9ca72dcc8c
paper_authors.avatarUrl: /avatars/b55101dc2b0052c36f0b2cbdea9473e7.svg
paper_authors.fullname: Filip Radenovic
paper_authors.type: user
paper_authors.user: filipradenovic
paper_authors._id: 651503184b4f53b8268ac648
paper_authors.name: Dhruv Mahajan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:44:39.690Z
paper_authors._id: 64358a51f81a16e7436330f7
paper_authors.avatarUrl: /avatars/71c350836e5d025b4c7b9f99f72c2e21.svg
paper_authors.fullname: Dhruv Mahajan
paper_authors.type: user
paper_authors.user: Dhruv2009
paper_authors._id: 651503184b4f53b8268ac649
paper_authors.name: Kunpeng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:44:47.175Z
paper_authors._id: 6418098e927a8ce346e40633
paper_authors.avatarUrl: /avatars/3b0e81e24e23db21b52c0a90c57faa8f.svg
paper_authors.fullname: Kunpeng Li
paper_authors.type: user
paper_authors.user: lkp
paper_authors._id: 651503184b4f53b8268ac64a
paper_authors.name: Yue Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-28T19:03:37.634Z
paper_authors._id: 641cc612f0b71a9743668604
paper_authors.avatarUrl: /avatars/ab7d91df8de266fadb4e5632a1f0a37a.svg
paper_authors.fullname: Yue Zhao
paper_authors.type: user
paper_authors.user: yzhao30
paper_authors._id: 651503184b4f53b8268ac64b
paper_authors.name: Vladan Petrovic
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-28T19:03:22.005Z
paper_authors._id: 6515caf71e7b9224c9d23851
paper_authors.avatarUrl: /avatars/11b17ba6d4f8e7b97a1644cc551a9fe7.svg
paper_authors.fullname: Vladan Petrovic
paper_authors.type: user
paper_authors.user: vpetrovi
paper_authors._id: 651503184b4f53b8268ac64c
paper_authors.name: Mitesh Kumar Singh
paper_authors._id: 651503184b4f53b8268ac64d
paper_authors.name: Simran Motwani
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-28T19:03:34.603Z
paper_authors._id: 6515bb7e99e8060f1e792047
paper_authors.avatarUrl: /avatars/9c6400f83c3165de3154b0e6bee2e5a1.svg
paper_authors.fullname: Simran Motwani
paper_authors.type: user
paper_authors.user: smotwani
paper_authors._id: 651503184b4f53b8268ac64e
paper_authors.name: Yi Wen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:53:52.586Z
paper_authors._id: 64519584938967fd069f7f45
paper_authors.avatarUrl: /avatars/4c5ebff5df4d290873fd6901e00a67e4.svg
paper_authors.fullname: Yi Wen
paper_authors.type: user
paper_authors.user: w-yi
paper_authors._id: 651503184b4f53b8268ac64f
paper_authors.name: Yiwen Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-02T07:00:16.343Z
paper_authors._id: 6517326e335b4c728b3406a4
paper_authors.avatarUrl: /avatars/cde4a672778fc69c172e6bdeee2d540a.svg
paper_authors.fullname: Yiwen Song
paper_authors.type: user
paper_authors.user: yiwen-song
paper_authors._id: 651503184b4f53b8268ac650
paper_authors.name: Roshan Sumbaly
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-28T19:03:25.119Z
paper_authors._id: 6515c595f464644a73b50064
paper_authors.avatarUrl: /avatars/119102c41fb8a7986c72b3f55ec6200f.svg
paper_authors.fullname: Roshan sumbaly
paper_authors.type: user
paper_authors.user: rsumbaly
paper_authors._id: 651503184b4f53b8268ac651
paper_authors.name: Vignesh Ramanathan
paper_authors._id: 651503184b4f53b8268ac652
paper_authors.name: Zijian He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-15T07:42:23.153Z
paper_authors._id: 657cfa56504da7f6f393c7a1
paper_authors.avatarUrl: /avatars/6add66f88a876147ef9a25c34ea6cf63.svg
paper_authors.fullname: zijian
paper_authors.type: user
paper_authors.user: Maxpooling
paper_authors._id: 651503184b4f53b8268ac653
paper_authors.name: Peter Vajda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:46:44.373Z
paper_authors._id: 636f4b35af79dd193603342b
paper_authors.avatarUrl: /avatars/577328195738d92ac41a3d0ff3108d7c.svg
paper_authors.fullname: Peter Vajda
paper_authors.type: user
paper_authors.user: vaphab
paper_authors._id: 651503184b4f53b8268ac654
paper_authors.name: Devi Parikh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-28T08:46:52.374Z
paper_authors._id: 6340753995c20b9447379c67
paper_authors.avatarUrl: /avatars/f793c00a98335cd8f7efce41d403da16.svg
paper_authors.fullname: Devi Parikh
paper_authors.type: user
paper_authors.user: deviparikh-genai
paper_title: Emu: Enhancing Image Generation Models Using Photogenic Needles in a
    Haystack
paper_summary: Training text-to-image models with web scale image-text pairs enables the
  generation of a wide range of visual concepts from text. However, these
  pre-trained models often face challenges when it comes to generating highly
  aesthetic images. This creates the need for aesthetic alignment post
  pre-training. In this paper, we propose quality-tuning to effectively guide a
  pre-trained model to exclusively generate highly visually appealing images,
  while maintaining generality across visual concepts. Our key insight is that
  supervised fine-tuning with a set of surprisingly small but extremely visually
  appealing images can significantly improve the generation quality. We pre-train
  a latent diffusion model on 1.1 billion image-text pairs and fine-tune it
  with only a few thousand carefully selected high-quality images. The resulting
  model, Emu, achieves a win rate of 82.9% compared with its pre-trained only
  counterpart. Compared to the state-of-the-art SDXLv1.0, Emu is preferred
  68.4% and 71.3% of the time on visual appeal on the standard PartiPrompts
  and our Open User Input benchmark based on the real-world usage of
  text-to-image models. In addition, we show that quality-tuning is a generic
  approach that is also effective for other architectures, including pixel
  diffusion and masked generative transformer models.
paper_ai_summary: Quality-tuning enhances the aesthetic appeal of text-to-image models through supervised fine-tuning with a small set of high-quality images.
paper_ai_keywords: latent diffusion model quality-tuning text-to-image models visual appeal supervised fine-tuning Emu SDXLv1.0 PartiPrompts Open User Input benchmark pixel diffusion masked generative transformer models