title: DiffMorpher: Unleashing the Capability of Diffusion Models for Image
    Morphing
summary: Diffusion models have achieved remarkable image generation quality surpassing
  previous generative models. However, a notable limitation of diffusion models,
  in comparison to GANs, is their difficulty in smoothly interpolating between
  two image samples, due to their highly unstructured latent space. Such a smooth
  interpolation is intriguing as it naturally serves as a solution for the image
  morphing task with many applications. In this work, we present DiffMorpher, the
  first approach enabling smooth and natural image interpolation using diffusion
  models. Our key idea is to capture the semantics of the two images by fitting
  two LoRAs to them respectively, and interpolate between both the LoRA
  parameters and the latent noises to ensure a smooth semantic transition, where
  correspondence automatically emerges without the need for annotation. In
  addition, we propose an attention interpolation and injection technique and a
  new sampling schedule to further enhance the smoothness between consecutive
  images. Extensive experiments demonstrate that DiffMorpher achieves starkly
  better image morphing effects than previous methods across a variety of object
  categories, bridging a critical functional gap that distinguished diffusion
  models from GANs.
paper_authors._id: 657925a04d989b0a68bfbafe
paper_authors.name: Kaiwen Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-13T08:36:55.843Z
paper_authors._id: 641087ddf52d7eb22e0673dd
paper_authors.avatarUrl: /avatars/946f29569aeda2cbd9ef2b94b1ab1391.svg
paper_authors.fullname: Kevin Zhang
paper_authors.type: user
paper_authors.user: Kevin-thu
paper_authors._id: 657925a04d989b0a68bfbaff
paper_authors.name: Yifan Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-02T12:52:47.947Z
paper_authors._id: 64a63f9449b08110f761cd73
paper_authors.avatarUrl: /avatars/61860202fc818b105ef24e74dd4f7d3c.svg
paper_authors.fullname: Yifan Zhou
paper_authors.type: user
paper_authors.user: SingleZombie
paper_authors._id: 657925a04d989b0a68bfbb00
paper_authors.name: Xudong Xu
paper_authors._id: 657925a04d989b0a68bfbb01
paper_authors.name: Xingang Pan
paper_authors._id: 657925a04d989b0a68bfbb02
paper_authors.name: Bo Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-13T08:40:01.097Z
paper_authors._id: 635f93577c05eb9f59966209
paper_authors.avatarUrl: /avatars/add48d7e3790a6b500d6c451ef8b0f75.svg
paper_authors.fullname: Intelligent Digital Creation
paper_authors.type: user
paper_authors.user: BoDai
paper_title: DiffMorpher: Unleashing the Capability of Diffusion Models for Image
    Morphing
paper_summary: Diffusion models have achieved remarkable image generation quality surpassing
  previous generative models. However, a notable limitation of diffusion models,
  in comparison to GANs, is their difficulty in smoothly interpolating between
  two image samples, due to their highly unstructured latent space. Such a smooth
  interpolation is intriguing as it naturally serves as a solution for the image
  morphing task with many applications. In this work, we present DiffMorpher, the
  first approach enabling smooth and natural image interpolation using diffusion
  models. Our key idea is to capture the semantics of the two images by fitting
  two LoRAs to them respectively, and interpolate between both the LoRA
  parameters and the latent noises to ensure a smooth semantic transition, where
  correspondence automatically emerges without the need for annotation. In
  addition, we propose an attention interpolation and injection technique and a
  new sampling schedule to further enhance the smoothness between consecutive
  images. Extensive experiments demonstrate that DiffMorpher achieves starkly
  better image morphing effects than previous methods across a variety of object
  categories, bridging a critical functional gap that distinguished diffusion
  models from GANs.
paper_ai_keywords: diffusion models image generation GANs smooth interpolation latent space image morphing LoRAs latent noises semantic transition attention interpolation sampling schedule