title: Tuning-Free Noise Rectification for High Fidelity Image-to-Video
    Generation
summary: Image-to-video (I2V) generation tasks always suffer from keeping high
  fidelity in the open domains. Traditional image animation techniques primarily
  focus on specific domains such as faces or human poses, making them difficult
  to generalize to open domains. Several recent I2V frameworks based on diffusion
  models can generate dynamic content for open domain images but fail to maintain
  fidelity. We found that two main factors of low fidelity are the loss of image
  details and the noise prediction biases during the denoising process. To this
  end, we propose an effective method that can be applied to mainstream video
  diffusion models. This method achieves high fidelity based on supplementing
  more precise image information and noise rectification. Specifically, given a
  specified image, our method first adds noise to the input image latent to keep
  more details, then denoises the noisy latent with proper rectification to
  alleviate the noise prediction biases. Our method is tuning-free and
  plug-and-play. The experimental results demonstrate the effectiveness of our
  approach in improving the fidelity of generated videos. For more image-to-video
  generated results, please refer to the project website:
  https://noise-rectification.github.io.
paper_authors._id: 65e7d0103fb61ba9c610aedc
paper_authors.name: Weijie Li
paper_authors._id: 65e7d0103fb61ba9c610aedd
paper_authors.name: Litong Gong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T10:40:55.980Z
paper_authors._id: 6322bc68a97afe3c4c6dd8a7
paper_authors.avatarUrl: /avatars/c014508235b57a9750621755efc189c6.svg
paper_authors.fullname: Litong.G
paper_authors.type: user
paper_authors.user: culeao
paper_authors._id: 65e7d0103fb61ba9c610aede
paper_authors.name: Yiran Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T11:38:37.782Z
paper_authors._id: 63ecce0054bba678151814cf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676463764139-63ecce0054bba678151814cf.png
paper_authors.fullname: y1Ran
paper_authors.type: user
paper_authors.user: y1Ran
paper_authors._id: 65e7d0103fb61ba9c610aedf
paper_authors.name: Fanda Fan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-06T10:41:31.788Z
paper_authors._id: 650152bed522560505e0e092
paper_authors.avatarUrl: /avatars/9fb3eee1b23859bfae6ca7d3c15e4bff.svg
paper_authors.fullname: Fanda Fan
paper_authors.type: user
paper_authors.user: stevenfan
paper_authors._id: 65e7d0103fb61ba9c610aee0
paper_authors.name: Biao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-06T15:34:27.063Z
paper_authors._id: 64dcd9de0a0efd28e3b846e3
paper_authors.avatarUrl: /avatars/0732be326b1d3658118f669dd283d582.svg
paper_authors.fullname: ictwangbiao
paper_authors.type: user
paper_authors.user: ictwangbiao
paper_authors._id: 65e7d0103fb61ba9c610aee1
paper_authors.name: Tiezheng Ge
paper_authors._id: 65e7d0103fb61ba9c610aee2
paper_authors.name: Bo Zheng
paper_title: Tuning-Free Noise Rectification for High Fidelity Image-to-Video
    Generation
paper_summary: Image-to-video (I2V) generation tasks always suffer from keeping high
  fidelity in the open domains. Traditional image animation techniques primarily
  focus on specific domains such as faces or human poses, making them difficult
  to generalize to open domains. Several recent I2V frameworks based on diffusion
  models can generate dynamic content for open domain images but fail to maintain
  fidelity. We found that two main factors of low fidelity are the loss of image
  details and the noise prediction biases during the denoising process. To this
  end, we propose an effective method that can be applied to mainstream video
  diffusion models. This method achieves high fidelity based on supplementing
  more precise image information and noise rectification. Specifically, given a
  specified image, our method first adds noise to the input image latent to keep
  more details, then denoises the noisy latent with proper rectification to
  alleviate the noise prediction biases. Our method is tuning-free and
  plug-and-play. The experimental results demonstrate the effectiveness of our
  approach in improving the fidelity of generated videos. For more image-to-video
  generated results, please refer to the project website:
  https://noise-rectification.github.io.
paper_ai_keywords: diffusion models denoising noise prediction biases image-to-video (I2V) image latent noise rectification high fidelity noise prediction biases neural execution