title: Denoising Reuse: Exploiting Inter-frame Motion Consistency for Efficient
    Video Latent Generation
summary: Video generation using diffusion-based models is constrained by high
  computational costs due to the frame-wise iterative diffusion process. This
  work presents a Diffusion Reuse MOtion (Dr. Mo) network to accelerate latent
  video generation. Our key discovery is that coarse-grained noises in earlier
  denoising steps have demonstrated high motion consistency across consecutive
  video frames. Following this observation, Dr. Mo propagates those
  coarse-grained noises onto the next frame by incorporating carefully designed,
  lightweight inter-frame motions, eliminating massive computational redundancy
  in frame-wise diffusion models. The more sensitive and fine-grained noises are
  still acquired via later denoising steps, which can be essential to retain
  visual qualities. As such, deciding which intermediate steps should switch from
  motion-based propagations to denoising can be a crucial problem and a key
  tradeoff between efficiency and quality. Dr. Mo employs a meta-network named
  Denoising Step Selector (DSS) to dynamically determine desirable intermediate
  steps across video frames. Extensive evaluations on video generation and
  editing tasks have shown that Dr. Mo can substantially accelerate diffusion
  models in video tasks with improved visual qualities.
paper_authors._id: 66ecd648dca68f236c888ecc
paper_authors.name: Chenyu Wang
paper_authors._id: 66ecd648dca68f236c888ecd
paper_authors.name: Shuo Yan
paper_authors._id: 66ecd648dca68f236c888ece
paper_authors.name: Yixuan Chen
paper_authors._id: 66ecd648dca68f236c888ecf
paper_authors.name: Yujiang Wang
paper_authors._id: 66ecd648dca68f236c888ed0
paper_authors.name: Mingzhi Dong
paper_authors._id: 66ecd648dca68f236c888ed1
paper_authors.name: Xiaochen Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T09:41:31.943Z
paper_authors._id: 6654140eff66ec2867458ced
paper_authors.avatarUrl: /avatars/6c34055ea1aebe015ac4a9b85969e94d.svg
paper_authors.fullname: Xiaochen Yang
paper_authors.type: user
paper_authors.user: genye
paper_authors._id: 66ecd648dca68f236c888ed2
paper_authors.name: Dongsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-20T09:42:14.989Z
paper_authors._id: 6524f13821af4b07309015aa
paper_authors.avatarUrl: /avatars/f97d6a041428d48305568df7afd83cc8.svg
paper_authors.fullname: dongsheng li
paper_authors.type: user
paper_authors.user: dongshengli
paper_authors._id: 66ecd648dca68f236c888ed3
paper_authors.name: Robert P. Dick
paper_authors._id: 66ecd648dca68f236c888ed4
paper_authors.name: Qin Lv
paper_authors._id: 66ecd648dca68f236c888ed5
paper_authors.name: Fan Yang
paper_authors._id: 66ecd648dca68f236c888ed6
paper_authors.name: Tun Lu
paper_authors._id: 66ecd648dca68f236c888ed7
paper_authors.name: Ning Gu
paper_authors._id: 66ecd648dca68f236c888ed8
paper_authors.name: Li Shang
paper_title: Denoising Reuse: Exploiting Inter-frame Motion Consistency for Efficient
    Video Latent Generation
paper_summary: Video generation using diffusion-based models is constrained by high
  computational costs due to the frame-wise iterative diffusion process. This
  work presents a Diffusion Reuse MOtion (Dr. Mo) network to accelerate latent
  video generation. Our key discovery is that coarse-grained noises in earlier
  denoising steps have demonstrated high motion consistency across consecutive
  video frames. Following this observation, Dr. Mo propagates those
  coarse-grained noises onto the next frame by incorporating carefully designed,
  lightweight inter-frame motions, eliminating massive computational redundancy
  in frame-wise diffusion models. The more sensitive and fine-grained noises are
  still acquired via later denoising steps, which can be essential to retain
  visual qualities. As such, deciding which intermediate steps should switch from
  motion-based propagations to denoising can be a crucial problem and a key
  tradeoff between efficiency and quality. Dr. Mo employs a meta-network named
  Denoising Step Selector (DSS) to dynamically determine desirable intermediate
  steps across video frames. Extensive evaluations on video generation and
  editing tasks have shown that Dr. Mo can substantially accelerate diffusion
  models in video tasks with improved visual qualities.
paper_ai_keywords: diffusion-based models computational costs frame-wise iterative diffusion process Diffusion Reuse MOtion (Dr. Mo) network coarse-grained noises denoising steps motion consistency consecutive video frames lightweight inter-frame motions fine-grained noises visual qualities Denoising Step Selector (DSS)