title: Generative Photomontage
summary: Text-to-image models are powerful tools for image creation. However, the
  generation process is akin to a dice roll and makes it difficult to achieve a
  single image that captures everything a user wants. In this paper, we propose a
  framework for creating the desired image by compositing it from various parts
  of generated images, in essence forming a Generative Photomontage. Given a
  stack of images generated by ControlNet using the same input condition and
  different seeds, we let users select desired parts from the generated results
  using a brush stroke interface. We introduce a novel technique that takes in
  the user's brush strokes, segments the generated images using a graph-based
  optimization in diffusion feature space, and then composites the segmented
  regions via a new feature-space blending method. Our method faithfully
  preserves the user-selected regions while compositing them harmoniously. We
  demonstrate that our flexible framework can be used for many applications,
  including generating new appearance combinations, fixing incorrect shapes and
  artifacts, and improving prompt alignment. We show compelling results for each
  application and demonstrate that our method outperforms existing image blending
  methods and various baselines.
paper_authors._id: 66bd60df453a7ef6c5cca4c7
paper_authors.name: Sean J. Liu
paper_authors._id: 66bd60df453a7ef6c5cca4c8
paper_authors.name: Nupur Kumari
paper_authors._id: 66bd60df453a7ef6c5cca4c9
paper_authors.name: Ariel Shamir
paper_authors._id: 66bd60df453a7ef6c5cca4ca
paper_authors.name: Jun-Yan Zhu
paper_title: Generative Photomontage
paper_summary: Text-to-image models are powerful tools for image creation. However, the
  generation process is akin to a dice roll and makes it difficult to achieve a
  single image that captures everything a user wants. In this paper, we propose a
  framework for creating the desired image by compositing it from various parts
  of generated images, in essence forming a Generative Photomontage. Given a
  stack of images generated by ControlNet using the same input condition and
  different seeds, we let users select desired parts from the generated results
  using a brush stroke interface. We introduce a novel technique that takes in
  the user's brush strokes, segments the generated images using a graph-based
  optimization in diffusion feature space, and then composites the segmented
  regions via a new feature-space blending method. Our method faithfully
  preserves the user-selected regions while compositing them harmoniously. We
  demonstrate that our flexible framework can be used for many applications,
  including generating new appearance combinations, fixing incorrect shapes and
  artifacts, and improving prompt alignment. We show compelling results for each
  application and demonstrate that our method outperforms existing image blending
  methods and various baselines.
paper_ai_summary: A framework using Generative Photomontage allows users to composite images from different parts of generated images, enhancing flexibility and quality in text-to-image models with new blending techniques.
paper_ai_keywords: ControlNet brush stroke interface graph-based optimization diffusion feature space feature-space blending prompt alignment