Composite Diffusion | whole >= Σparts
For an artist or a graphic designer, the spatial layout of a scene is a
critical design choice. However, existing text-to-image diffusion models
provide limited support for incorporating spatial information. This paper
introduces Composite Diffusion as a means for artists to generate high-quality
images by composing from the sub-scenes. The artists can specify the
arrangement of these sub-scenes through a flexible free-form segment layout.
They can describe the content of each sub-scene primarily using natural text
and additionally by utilizing reference images or control inputs such as line
art, scribbles, human pose, canny edges, and more.
  We provide a comprehensive and modular method for Composite Diffusion that
enables alternative ways of generating, composing, and harmonizing sub-scenes.
Further, we wish to evaluate the composite image for effectiveness in both
image quality and achieving the artist's intent. We argue that existing image
quality metrics lack a holistic evaluation of image composites. To address
this, we propose novel quality criteria especially relevant to composite
generation.
  We believe that our approach provides an intuitive method of art creation.
Through extensive user surveys, quantitative and qualitative analysis, we show
how it achieves greater spatial, semantic, and creative control over image
generation. In addition, our methods do not need to retrain or modify the
architecture of the base diffusion models and can work in a plug-and-play
manner with the fine-tuned models.
64c1c980203c836866dea993
Vikram Jamwal
claimed_verified
2023-08-01T07:44:58.460Z
641ad60fec5b871c0bcaf2e6
/avatars/b01dcf8d8e4c65c12e4573b43f76c909.svg
Vikram Jamwal
user
vikram-tcs
64c1c980203c836866dea994
Ramaneswaran S
claimed_verified
2023-07-27T17:32:43.831Z
61421377db4ebda731f4f0b4
https://cdn-avatars.huggingface.co/v1/production/uploads/1631720286113-noauth.jpeg
Raman
user
TheDetective
Composite Diffusion | whole >= Σparts
For an artist or a graphic designer, the spatial layout of a scene is a
critical design choice. However, existing text-to-image diffusion models
provide limited support for incorporating spatial information. This paper
introduces Composite Diffusion as a means for artists to generate high-quality
images by composing from the sub-scenes. The artists can specify the
arrangement of these sub-scenes through a flexible free-form segment layout.
They can describe the content of each sub-scene primarily using natural text
and additionally by utilizing reference images or control inputs such as line
art, scribbles, human pose, canny edges, and more.
  We provide a comprehensive and modular method for Composite Diffusion that
enables alternative ways of generating, composing, and harmonizing sub-scenes.
Further, we wish to evaluate the composite image for effectiveness in both
image quality and achieving the artist's intent. We argue that existing image
quality metrics lack a holistic evaluation of image composites. To address
this, we propose novel quality criteria especially relevant to composite
generation.
  We believe that our approach provides an intuitive method of art creation.
Through extensive user surveys, quantitative and qualitative analysis, we show
how it achieves greater spatial, semantic, and creative control over image
generation. In addition, our methods do not need to retrain or modify the
architecture of the base diffusion models and can work in a plug-and-play
manner with the fine-tuned models.
Composite Diffusion enhances text-to-image generation by composing sub-scenes with spatial control, using novel quality criteria.
Composite Diffusion sub-scenes text-to-image diffusion models free-form segment layout reference images control inputs human pose canny edges modular method image composites image quality metrics intuitive art creation spatial control semantic control creative control plug-and-play fine-tuned models