title: SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions
summary: The remarkable capabilities of pretrained image diffusion models have been
  utilized not only for generating fixed-size images but also for creating
  panoramas. However, naive stitching of multiple images often results in visible
  seams. Recent techniques have attempted to address this issue by performing
  joint diffusions in multiple windows and averaging latent features in
  overlapping regions. However, these approaches, which focus on seamless montage
  generation, often yield incoherent outputs by blending different scenes within
  a single image. To overcome this limitation, we propose SyncDiffusion, a
  plug-and-play module that synchronizes multiple diffusions through gradient
  descent from a perceptual similarity loss. Specifically, we compute the
  gradient of the perceptual loss using the predicted denoised images at each
  denoising step, providing meaningful guidance for achieving coherent montages.
  Our experimental results demonstrate that our method produces significantly
  more coherent outputs compared to previous methods (66.35% vs. 33.65% in our
  user study) while still maintaining fidelity (as assessed by GIQA) and
  compatibility with the input prompt (as measured by CLIP score).
paper_authors._id: 6482b06a474d4d463d4c5342
paper_authors.name: Yuseung Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-12T09:12:20.366Z
paper_authors._id: 6342796a0875f2c99cfd313b
paper_authors.avatarUrl: /avatars/98575092404c4197b20c929a6499a015.svg
paper_authors.fullname: Yuseung "Phillip" Lee
paper_authors.type: user
paper_authors.user: phillipinseoul
paper_authors._id: 6482b06a474d4d463d4c5343
paper_authors.name: Kunho Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-12T13:27:21.011Z
paper_authors._id: 644b5c969279988e0cbecd56
paper_authors.avatarUrl: /avatars/be2983528677d88647e031538f3f6c40.svg
paper_authors.fullname: Kunho Kim
paper_authors.type: user
paper_authors.user: Kunho
paper_authors._id: 6482b06a474d4d463d4c5344
paper_authors.name: Hyunjin Kim
paper_authors._id: 6482b06a474d4d463d4c5345
paper_authors.name: Minhyuk Sung
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-12T13:27:45.261Z
paper_authors._id: 631f432b5ba8c026340a7890
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/631f432b5ba8c026340a7890/9PK7A_TRMpugwYjCsNBf1.jpeg
paper_authors.fullname: Minhyuk Sung
paper_authors.type: user
paper_authors.user: Minhyuk
paper_title: SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions
paper_summary: The remarkable capabilities of pretrained image diffusion models have been
  utilized not only for generating fixed-size images but also for creating
  panoramas. However, naive stitching of multiple images often results in visible
  seams. Recent techniques have attempted to address this issue by performing
  joint diffusions in multiple windows and averaging latent features in
  overlapping regions. However, these approaches, which focus on seamless montage
  generation, often yield incoherent outputs by blending different scenes within
  a single image. To overcome this limitation, we propose SyncDiffusion, a
  plug-and-play module that synchronizes multiple diffusions through gradient
  descent from a perceptual similarity loss. Specifically, we compute the
  gradient of the perceptual loss using the predicted denoised images at each
  denoising step, providing meaningful guidance for achieving coherent montages.
  Our experimental results demonstrate that our method produces significantly
  more coherent outputs compared to previous methods (66.35% vs. 33.65% in our
  user study) while still maintaining fidelity (as assessed by GIQA) and
  compatibility with the input prompt (as measured by CLIP score).
paper_ai_keywords: SyncDiffusion perceptual similarity loss gradient descent denoised images coherent montages GIQA CLIP score