title: Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesis
summary: In this paper, we introduce Fairy, a minimalist yet robust adaptation of
  image-editing diffusion models, enhancing them for video editing applications.
  Our approach centers on the concept of anchor-based cross-frame attention, a
  mechanism that implicitly propagates diffusion features across frames, ensuring
  superior temporal coherence and high-fidelity synthesis. Fairy not only
  addresses limitations of previous models, including memory and processing
  speed. It also improves temporal consistency through a unique data augmentation
  strategy. This strategy renders the model equivariant to affine transformations
  in both source and target images. Remarkably efficient, Fairy generates
  120-frame 512x384 videos (4-second duration at 30 FPS) in just 14 seconds,
  outpacing prior works by at least 44x. A comprehensive user study, involving
  1000 generated samples, confirms that our approach delivers superior quality,
  decisively outperforming established methods.
paper_authors._id: 6584f1bfebe2be20806e3e51
paper_authors.name: Bichen Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T08:25:31.748Z
paper_authors._id: 6363749520c2cf37bccd9e84
paper_authors.avatarUrl: /avatars/e9de7911e67090d883f15a7a58f18e4e.svg
paper_authors.fullname: Bichen Wu
paper_authors.type: user
paper_authors.user: bichenwu
paper_authors._id: 6584f1bfebe2be20806e3e52
paper_authors.name: Ching-Yao Chuang
paper_authors._id: 6584f1bfebe2be20806e3e53
paper_authors.name: Xiaoyan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T08:28:22.573Z
paper_authors._id: 63a50790e3623e66842f5fc8
paper_authors.avatarUrl: /avatars/4cc919b7b9fd393595599b4897a75e05.svg
paper_authors.fullname: Xiaoyan Wang
paper_authors.type: user
paper_authors.user: Horizonian
paper_authors._id: 6584f1bfebe2be20806e3e54
paper_authors.name: Yichen Jia
paper_authors._id: 6584f1bfebe2be20806e3e55
paper_authors.name: Kapil Krishnakumar
paper_authors._id: 6584f1bfebe2be20806e3e56
paper_authors.name: Tong Xiao
paper_authors._id: 6584f1bfebe2be20806e3e57
paper_authors.name: Feng Liang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-02T09:27:37.952Z
paper_authors._id: 63631be58f43a912fc7595d5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1667440137801-63631be58f43a912fc7595d5.jpeg
paper_authors.fullname: Jeff Liang
paper_authors.type: user
paper_authors.user: JeffLiang
paper_authors._id: 6584f1bfebe2be20806e3e58
paper_authors.name: Licheng Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T08:26:55.357Z
paper_authors._id: 6372b1f9bd81fae2b3a712e8
paper_authors.avatarUrl: /avatars/bcdf7b540cc94882e0ae6035bb07a8f2.svg
paper_authors.fullname: Licheng Yu
paper_authors.type: user
paper_authors.user: lichengyu
paper_authors._id: 6584f1bfebe2be20806e3e59
paper_authors.name: Peter Vajda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T11:02:07.614Z
paper_authors._id: 636f4b35af79dd193603342b
paper_authors.avatarUrl: /avatars/577328195738d92ac41a3d0ff3108d7c.svg
paper_authors.fullname: Peter Vajda
paper_authors.type: user
paper_authors.user: vaphab
paper_title: Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesis
paper_summary: In this paper, we introduce Fairy, a minimalist yet robust adaptation of
  image-editing diffusion models, enhancing them for video editing applications.
  Our approach centers on the concept of anchor-based cross-frame attention, a
  mechanism that implicitly propagates diffusion features across frames, ensuring
  superior temporal coherence and high-fidelity synthesis. Fairy not only
  addresses limitations of previous models, including memory and processing
  speed. It also improves temporal consistency through a unique data augmentation
  strategy. This strategy renders the model equivariant to affine transformations
  in both source and target images. Remarkably efficient, Fairy generates
  120-frame 512x384 videos (4-second duration at 30 FPS) in just 14 seconds,
  outpacing prior works by at least 44x. A comprehensive user study, involving
  1000 generated samples, confirms that our approach delivers superior quality,
  decisively outperforming established methods.
paper_ai_keywords: diffusion models