title: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two
    Seconds
summary: Text-to-image diffusion models can create stunning images from natural
  language descriptions that rival the work of professional artists and
  photographers. However, these models are large, with complex network
  architectures and tens of denoising iterations, making them computationally
  expensive and slow to run. As a result, high-end GPUs and cloud-based inference
  are required to run diffusion models at scale. This is costly and has privacy
  implications, especially when user data is sent to a third party. To overcome
  these challenges, we present a generic approach that, for the first time,
  unlocks running text-to-image diffusion models on mobile devices in less than
  2 seconds. We achieve so by introducing efficient network architecture and
  improving step distillation. Specifically, we propose an efficient UNet by
  identifying the redundancy of the original model and reducing the computation
  of the image decoder via data distillation. Further, we enhance the step
  distillation by exploring training strategies and introducing regularization
  from classifier-free guidance. Our extensive experiments on MS-COCO show that
  our model with 8 denoising steps achieves better FID and CLIP scores than
  Stable Diffusion v1.5 with 50 steps. Our work democratizes content creation
  by bringing powerful text-to-image diffusion models to the hands of users.
paper_authors._id: 64797f1b12ae749b62ee7671
paper_authors.name: Yanyu Li
paper_authors._id: 64797f1b12ae749b62ee7672
paper_authors.name: Huan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-02T12:46:32.679Z
paper_authors._id: 62b624f3b52bef716e248fd7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b624f3b52bef716e248fd7/AllcccKH-eBWduA8KVnOQ.png
paper_authors.fullname: Huan Wang
paper_authors.type: user
paper_authors.user: Huan-WhoRegisteredMyName
paper_authors._id: 64797f1b12ae749b62ee7673
paper_authors.name: Qing Jin
paper_authors._id: 64797f1b12ae749b62ee7674
paper_authors.name: Ju Hu
paper_authors._id: 64797f1b12ae749b62ee7675
paper_authors.name: Pavlo Chemerys
paper_authors._id: 64797f1b12ae749b62ee7676
paper_authors.name: Yun Fu
paper_authors._id: 64797f1b12ae749b62ee7677
paper_authors.name: Yanzhi Wang
paper_authors._id: 64797f1b12ae749b62ee7678
paper_authors.name: Sergey Tulyakov
paper_authors._id: 64797f1b12ae749b62ee7679
paper_authors.name: Jian Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-05T10:08:00.227Z
paper_authors._id: 61f19829233c91cbd2f79e70
paper_authors.avatarUrl: /avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
paper_authors.fullname: Jian Ren
paper_authors.type: user
paper_authors.user: alanspike
paper_title: SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two
    Seconds
paper_summary: Text-to-image diffusion models can create stunning images from natural
  language descriptions that rival the work of professional artists and
  photographers. However, these models are large, with complex network
  architectures and tens of denoising iterations, making them computationally
  expensive and slow to run. As a result, high-end GPUs and cloud-based inference
  are required to run diffusion models at scale. This is costly and has privacy
  implications, especially when user data is sent to a third party. To overcome
  these challenges, we present a generic approach that, for the first time,
  unlocks running text-to-image diffusion models on mobile devices in less than
  2 seconds. We achieve so by introducing efficient network architecture and
  improving step distillation. Specifically, we propose an efficient UNet by
  identifying the redundancy of the original model and reducing the computation
  of the image decoder via data distillation. Further, we enhance the step
  distillation by exploring training strategies and introducing regularization
  from classifier-free guidance. Our extensive experiments on MS-COCO show that
  our model with 8 denoising steps achieves better FID and CLIP scores than
  Stable Diffusion v1.5 with 50 steps. Our work democratizes content creation
  by bringing powerful text-to-image diffusion models to the hands of users.
paper_ai_keywords: text-to-image diffusion models denoising iterations high-end GPUs cloud-based inference mobile devices efficient network architecture step distillation efficient UNet data distillation classifier-free guidance MS-COCO FID CLIP scores Stable Diffusion