title: Composite Diffusion | whole >= Σparts
summary: For an artist or a graphic designer, the spatial layout of a scene is a
  critical design choice. However, existing text-to-image diffusion models
  provide limited support for incorporating spatial information. This paper
  introduces Composite Diffusion as a means for artists to generate high-quality
  images by composing from the sub-scenes. The artists can specify the
  arrangement of these sub-scenes through a flexible free-form segment layout.
  They can describe the content of each sub-scene primarily using natural text
  and additionally by utilizing reference images or control inputs such as line
  art, scribbles, human pose, canny edges, and more.
    We provide a comprehensive and modular method for Composite Diffusion that
  enables alternative ways of generating, composing, and harmonizing sub-scenes.
  Further, we wish to evaluate the composite image for effectiveness in both
  image quality and achieving the artist's intent. We argue that existing image
  quality metrics lack a holistic evaluation of image composites. To address
  this, we propose novel quality criteria especially relevant to composite
  generation.
    We believe that our approach provides an intuitive method of art creation.
  Through extensive user surveys, quantitative and qualitative analysis, we show
  how it achieves greater spatial, semantic, and creative control over image
  generation. In addition, our methods do not need to retrain or modify the
  architecture of the base diffusion models and can work in a plug-and-play
  manner with the fine-tuned models.
paper_authors._id: 64c1c980203c836866dea993
paper_authors.name: Vikram Jamwal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-01T07:44:58.460Z
paper_authors._id: 641ad60fec5b871c0bcaf2e6
paper_authors.avatarUrl: /avatars/b01dcf8d8e4c65c12e4573b43f76c909.svg
paper_authors.fullname: Vikram Jamwal
paper_authors.type: user
paper_authors.user: vikram-tcs
paper_authors._id: 64c1c980203c836866dea994
paper_authors.name: Ramaneswaran S
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-27T17:32:43.831Z
paper_authors._id: 61421377db4ebda731f4f0b4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1631720286113-noauth.jpeg
paper_authors.fullname: Raman
paper_authors.type: user
paper_authors.user: TheDetective
paper_title: Composite Diffusion | whole >= Σparts
paper_summary: For an artist or a graphic designer, the spatial layout of a scene is a
  critical design choice. However, existing text-to-image diffusion models
  provide limited support for incorporating spatial information. This paper
  introduces Composite Diffusion as a means for artists to generate high-quality
  images by composing from the sub-scenes. The artists can specify the
  arrangement of these sub-scenes through a flexible free-form segment layout.
  They can describe the content of each sub-scene primarily using natural text
  and additionally by utilizing reference images or control inputs such as line
  art, scribbles, human pose, canny edges, and more.
    We provide a comprehensive and modular method for Composite Diffusion that
  enables alternative ways of generating, composing, and harmonizing sub-scenes.
  Further, we wish to evaluate the composite image for effectiveness in both
  image quality and achieving the artist's intent. We argue that existing image
  quality metrics lack a holistic evaluation of image composites. To address
  this, we propose novel quality criteria especially relevant to composite
  generation.
    We believe that our approach provides an intuitive method of art creation.
  Through extensive user surveys, quantitative and qualitative analysis, we show
  how it achieves greater spatial, semantic, and creative control over image
  generation. In addition, our methods do not need to retrain or modify the
  architecture of the base diffusion models and can work in a plug-and-play
  manner with the fine-tuned models.
paper_ai_keywords: Composite Diffusion text-to-image diffusion models sub-scenes free-form segment layout natural text reference images line art scribbles human pose canny edges