title: RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths
summary: Text-to-image generation has recently witnessed remarkable achievements. We
  introduce a text-conditional image diffusion model, termed RAPHAEL, to generate
  highly artistic images, which accurately portray the text prompts, encompassing
  multiple nouns, adjectives, and verbs. This is achieved by stacking tens of
  mixture-of-experts (MoEs) layers, i.e., space-MoE and time-MoE layers, enabling
  billions of diffusion paths (routes) from the network input to the output. Each
  path intuitively functions as a "painter" for depicting a particular textual
  concept onto a specified image region at a diffusion timestep. Comprehensive
  experiments reveal that RAPHAEL outperforms recent cutting-edge models, such as
  Stable Diffusion, ERNIE-ViLG 2.0, DeepFloyd, and DALL-E 2, in terms of both
  image quality and aesthetic appeal. Firstly, RAPHAEL exhibits superior
  performance in switching images across diverse styles, such as Japanese comics,
  realism, cyberpunk, and ink illustration. Secondly, a single model with three
  billion parameters, trained on 1,000 A100 GPUs for two months, achieves a
  state-of-the-art zero-shot FID score of 6.61 on the COCO dataset. Furthermore,
  RAPHAEL significantly surpasses its counterparts in human evaluation on the
  ViLG-300 benchmark. We believe that RAPHAEL holds the potential to propel the
  frontiers of image generation research in both academia and industry, paving
  the way for future breakthroughs in this rapidly evolving field. More details
  can be found on a project webpage: https://raphael-painter.github.io/.
paper_authors._id: 64757135bb0e9dd7764b12ec
paper_authors.name: Zeyue Xue
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-30T14:15:14.392Z
paper_authors._id: 63721f5ada3183d9d53cfe1f
paper_authors.avatarUrl: /avatars/593c14c907848da7dbc9e5418751bd94.svg
paper_authors.fullname: Xue Zeyue
paper_authors.type: user
paper_authors.user: xzyhku
paper_authors._id: 64757135bb0e9dd7764b12ed
paper_authors.name: Guanglu Song
paper_authors._id: 64757135bb0e9dd7764b12ee
paper_authors.name: Qiushan Guo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-30T09:10:44.463Z
paper_authors._id: 64340ef7d55dea2d0ec0de6a
paper_authors.avatarUrl: /avatars/3baaa05090306910f77b368abcb27bdc.svg
paper_authors.fullname: qsguo
paper_authors.type: user
paper_authors.user: qiushanhku
paper_authors._id: 64757135bb0e9dd7764b12ef
paper_authors.name: Boxiao Liu
paper_authors._id: 64757135bb0e9dd7764b12f0
paper_authors.name: Zhuofan Zong
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-18T13:43:24.047Z
paper_authors._id: 64db05f23725f8d9a9f566c4
paper_authors.avatarUrl: /avatars/345b6d2dfadb91ac1f02be023dd27c8f.svg
paper_authors.fullname: Zhuofan Zong
paper_authors.type: user
paper_authors.user: zongzhuofan
paper_authors._id: 64757135bb0e9dd7764b12f1
paper_authors.name: Yu Liu
paper_authors._id: 64757135bb0e9dd7764b12f2
paper_authors.name: Ping Luo
paper_title: RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths
paper_summary: Text-to-image generation has recently witnessed remarkable achievements. We
  introduce a text-conditional image diffusion model, termed RAPHAEL, to generate
  highly artistic images, which accurately portray the text prompts, encompassing
  multiple nouns, adjectives, and verbs. This is achieved by stacking tens of
  mixture-of-experts (MoEs) layers, i.e., space-MoE and time-MoE layers, enabling
  billions of diffusion paths (routes) from the network input to the output. Each
  path intuitively functions as a "painter" for depicting a particular textual
  concept onto a specified image region at a diffusion timestep. Comprehensive
  experiments reveal that RAPHAEL outperforms recent cutting-edge models, such as
  Stable Diffusion, ERNIE-ViLG 2.0, DeepFloyd, and DALL-E 2, in terms of both
  image quality and aesthetic appeal. Firstly, RAPHAEL exhibits superior
  performance in switching images across diverse styles, such as Japanese comics,
  realism, cyberpunk, and ink illustration. Secondly, a single model with three
  billion parameters, trained on 1,000 A100 GPUs for two months, achieves a
  state-of-the-art zero-shot FID score of 6.61 on the COCO dataset. Furthermore,
  RAPHAEL significantly surpasses its counterparts in human evaluation on the
  ViLG-300 benchmark. We believe that RAPHAEL holds the potential to propel the
  frontiers of image generation research in both academia and industry, paving
  the way for future breakthroughs in this rapidly evolving field. More details
  can be found on a project webpage: https://raphael-painter.github.io/.
paper_ai_keywords: text-conditional image diffusion model RAPHAEL mixture-of-experts (MoEs) space-MoE time-MoE diffusion paths textual concept image region diffusion timestep zero-shot FID score COCO dataset human evaluation ViLG-300 benchmark