title: Alias-Free Latent Diffusion Models:Improving Fractional Shift
    Equivariance of Diffusion Latent Space
summary: Latent Diffusion Models (LDMs) are known to have an unstable generation
  process, where even small perturbations or shifts in the input noise can lead
  to significantly different outputs. This hinders their applicability in
  applications requiring consistent results. In this work, we redesign LDMs to
  enhance consistency by making them shift-equivariant. While introducing
  anti-aliasing operations can partially improve shift-equivariance, significant
  aliasing and inconsistency persist due to the unique challenges in LDMs,
  including 1) aliasing amplification during VAE training and multiple U-Net
  inferences, and 2) self-attention modules that inherently lack
  shift-equivariance. To address these issues, we redesign the attention modules
  to be shift-equivariant and propose an equivariance loss that effectively
  suppresses the frequency bandwidth of the features in the continuous domain.
  The resulting alias-free LDM (AF-LDM) achieves strong shift-equivariance and is
  also robust to irregular warping. Extensive experiments demonstrate that AF-LDM
  produces significantly more consistent results than vanilla LDM across various
  applications, including video editing and image-to-image translation. Code is
  available at: https://github.com/SingleZombie/AFLDM
paper_authors._id: 67d27032825fbe674d2e4109
paper_authors.name: Yifan Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-13T08:24:00.485Z
paper_authors._id: 64a63f9449b08110f761cd73
paper_authors.avatarUrl: /avatars/61860202fc818b105ef24e74dd4f7d3c.svg
paper_authors.fullname: Yifan Zhou
paper_authors.type: user
paper_authors.user: SingleZombie
paper_authors._id: 67d27032825fbe674d2e410a
paper_authors.name: Zeqi Xiao
paper_authors._id: 67d27032825fbe674d2e410b
paper_authors.name: Shuai Yang
paper_authors._id: 67d27032825fbe674d2e410c
paper_authors.name: Xingang Pan
paper_title: Alias-Free Latent Diffusion Models:Improving Fractional Shift
    Equivariance of Diffusion Latent Space
paper_summary: Latent Diffusion Models (LDMs) are known to have an unstable generation
  process, where even small perturbations or shifts in the input noise can lead
  to significantly different outputs. This hinders their applicability in
  applications requiring consistent results. In this work, we redesign LDMs to
  enhance consistency by making them shift-equivariant. While introducing
  anti-aliasing operations can partially improve shift-equivariance, significant
  aliasing and inconsistency persist due to the unique challenges in LDMs,
  including 1) aliasing amplification during VAE training and multiple U-Net
  inferences, and 2) self-attention modules that inherently lack
  shift-equivariance. To address these issues, we redesign the attention modules
  to be shift-equivariant and propose an equivariance loss that effectively
  suppresses the frequency bandwidth of the features in the continuous domain.
  The resulting alias-free LDM (AF-LDM) achieves strong shift-equivariance and is
  also robust to irregular warping. Extensive experiments demonstrate that AF-LDM
  produces significantly more consistent results than vanilla LDM across various
  applications, including video editing and image-to-image translation. Code is
  available at: https://github.com/SingleZombie/AFLDM
paper_ai_keywords: Latent Diffusion Models (LDMs) shift-equivariant anti-aliasing operations aliasing amplification VAE training U-Net self-attention modules attention modules equivariance loss alias-free LDM (AF-LDM) video editing image-to-image translation