RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths
Text-to-image generation has recently witnessed remarkable achievements. We
introduce a text-conditional image diffusion model, termed RAPHAEL, to generate
highly artistic images, which accurately portray the text prompts, encompassing
multiple nouns, adjectives, and verbs. This is achieved by stacking tens of
mixture-of-experts (MoEs) layers, i.e., space-MoE and time-MoE layers, enabling
billions of diffusion paths (routes) from the network input to the output. Each
path intuitively functions as a "painter" for depicting a particular textual
concept onto a specified image region at a diffusion timestep. Comprehensive
experiments reveal that RAPHAEL outperforms recent cutting-edge models, such as
Stable Diffusion, ERNIE-ViLG 2.0, DeepFloyd, and DALL-E 2, in terms of both
image quality and aesthetic appeal. Firstly, RAPHAEL exhibits superior
performance in switching images across diverse styles, such as Japanese comics,
realism, cyberpunk, and ink illustration. Secondly, a single model with three
billion parameters, trained on 1,000 A100 GPUs for two months, achieves a
state-of-the-art zero-shot FID score of 6.61 on the COCO dataset. Furthermore,
RAPHAEL significantly surpasses its counterparts in human evaluation on the
ViLG-300 benchmark. We believe that RAPHAEL holds the potential to propel the
frontiers of image generation research in both academia and industry, paving
the way for future breakthroughs in this rapidly evolving field. More details
can be found on a project webpage: https://raphael-painter.github.io/.
64757135bb0e9dd7764b12ec
Zeyue Xue
claimed_verified
2023-05-30T14:15:14.392Z
63721f5ada3183d9d53cfe1f
/avatars/593c14c907848da7dbc9e5418751bd94.svg
Xue Zeyue
user
xzyhku
64757135bb0e9dd7764b12ed
Guanglu Song
64757135bb0e9dd7764b12ee
Qiushan Guo
claimed_verified
2023-05-30T09:10:44.463Z
64340ef7d55dea2d0ec0de6a
/avatars/3baaa05090306910f77b368abcb27bdc.svg
qsguo
user
qiushanhku
64757135bb0e9dd7764b12ef
Boxiao Liu
64757135bb0e9dd7764b12f0
Zhuofan Zong
extracted_confirmed
2024-06-18T13:43:24.047Z
64db05f23725f8d9a9f566c4
/avatars/345b6d2dfadb91ac1f02be023dd27c8f.svg
Zhuofan Zong
user
zongzhuofan
64757135bb0e9dd7764b12f1
Yu Liu
64757135bb0e9dd7764b12f2
Ping Luo
RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths
Text-to-image generation has recently witnessed remarkable achievements. We
introduce a text-conditional image diffusion model, termed RAPHAEL, to generate
highly artistic images, which accurately portray the text prompts, encompassing
multiple nouns, adjectives, and verbs. This is achieved by stacking tens of
mixture-of-experts (MoEs) layers, i.e., space-MoE and time-MoE layers, enabling
billions of diffusion paths (routes) from the network input to the output. Each
path intuitively functions as a "painter" for depicting a particular textual
concept onto a specified image region at a diffusion timestep. Comprehensive
experiments reveal that RAPHAEL outperforms recent cutting-edge models, such as
Stable Diffusion, ERNIE-ViLG 2.0, DeepFloyd, and DALL-E 2, in terms of both
image quality and aesthetic appeal. Firstly, RAPHAEL exhibits superior
performance in switching images across diverse styles, such as Japanese comics,
realism, cyberpunk, and ink illustration. Secondly, a single model with three
billion parameters, trained on 1,000 A100 GPUs for two months, achieves a
state-of-the-art zero-shot FID score of 6.61 on the COCO dataset. Furthermore,
RAPHAEL significantly surpasses its counterparts in human evaluation on the
ViLG-300 benchmark. We believe that RAPHAEL holds the potential to propel the
frontiers of image generation research in both academia and industry, paving
the way for future breakthroughs in this rapidly evolving field. More details
can be found on a project webpage: https://raphael-painter.github.io/.
A text-conditional image diffusion model named RAPHAEL generates high-quality and artistic images by stacking mixture-of-experts layers, outperforming existing models in style switching and zero-shot FID score.
text-conditional image diffusion model RAPHAEL mixture-of-experts space-MoE time-MoE diffusion paths COCO dataset zero-shot FID score ViLG-300 benchmark