title: VACE: All-in-One Video Creation and Editing
summary: Diffusion Transformer has demonstrated powerful capability and scalability in
  generating high-quality images and videos. Further pursuing the unification of
  generation and editing tasks has yielded significant progress in the domain of
  image content creation. However, due to the intrinsic demands for consistency
  across both temporal and spatial dynamics, achieving a unified approach for
  video synthesis remains challenging. We introduce VACE, which enables users to
  perform Video tasks within an All-in-one framework for Creation and Editing.
  These tasks include reference-to-video generation, video-to-video editing, and
  masked video-to-video editing. Specifically, we effectively integrate the
  requirements of various tasks by organizing video task inputs, such as editing,
  reference, and masking, into a unified interface referred to as the Video
  Condition Unit (VCU). Furthermore, by utilizing a Context Adapter structure, we
  inject different task concepts into the model using formalized representations
  of temporal and spatial dimensions, allowing it to handle arbitrary video
  synthesis tasks flexibly. Extensive experiments demonstrate that the unified
  model of VACE achieves performance on par with task-specific models across
  various subtasks. Simultaneously, it enables diverse applications through
  versatile task combinations. Project page:
  https://ali-vilab.github.io/VACE-Page/.
paper_authors._id: 67d04a5d7c367a40ae3e417a
paper_authors.name: Zeyinzi Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:42:12.022Z
paper_authors._id: 643d278b482011f5f2bd0fae
paper_authors.avatarUrl: /avatars/70b8a7ffbfa2a1c4b6f5ff5e2b96b7bf.svg
paper_authors.fullname: jiangzeyinzi
paper_authors.type: user
paper_authors.user: jiangzeyinzi
paper_authors._id: 67d04a5d7c367a40ae3e417b
paper_authors.name: Zhen Han
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:42:18.152Z
paper_authors._id: 647ffabf28b737d7b9462eb2
paper_authors.avatarUrl: /avatars/210441fc6645d08b36ad43734108f914.svg
paper_authors.fullname: Zhen Han
paper_authors.type: user
paper_authors.user: hanzhn
paper_authors._id: 67d04a5d7c367a40ae3e417c
paper_authors.name: Chaojie Mao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:42:14.936Z
paper_authors._id: 6458970cab9a44f42f620a80
paper_authors.avatarUrl: /avatars/f9779b0621c931f922440fec95342444.svg
paper_authors.fullname: chaojie mao
paper_authors.type: user
paper_authors.user: chaojiemao
paper_authors._id: 67d04a5d7c367a40ae3e417d
paper_authors.name: Jingfeng Zhang
paper_authors._id: 67d04a5d7c367a40ae3e417e
paper_authors.name: Yulin Pan
paper_authors._id: 67d04a5d7c367a40ae3e417f
paper_authors.name: Yu Liu
paper_title: VACE: All-in-One Video Creation and Editing
paper_summary: Diffusion Transformer has demonstrated powerful capability and scalability in
  generating high-quality images and videos. Further pursuing the unification of
  generation and editing tasks has yielded significant progress in the domain of
  image content creation. However, due to the intrinsic demands for consistency
  across both temporal and spatial dynamics, achieving a unified approach for
  video synthesis remains challenging. We introduce VACE, which enables users to
  perform Video tasks within an All-in-one framework for Creation and Editing.
  These tasks include reference-to-video generation, video-to-video editing, and
  masked video-to-video editing. Specifically, we effectively integrate the
  requirements of various tasks by organizing video task inputs, such as editing,
  reference, and masking, into a unified interface referred to as the Video
  Condition Unit (VCU). Furthermore, by utilizing a Context Adapter structure, we
  inject different task concepts into the model using formalized representations
  of temporal and spatial dimensions, allowing it to handle arbitrary video
  synthesis tasks flexibly. Extensive experiments demonstrate that the unified
  model of VACE achieves performance on par with task-specific models across
  various subtasks. Simultaneously, it enables diverse applications through
  versatile task combinations. Project page:
  https://ali-vilab.github.io/VACE-Page/.
paper_ai_summary: VACE, an all-in-one framework for video creation and editing, integrates multiple tasks within a unified model using a Video Condition Unit and Context Adapter for flexible and consistent video synthesis.
paper_ai_keywords: diffusion transformer video synthesis reference-to-video generation video-to-video editing masked video-to-video editing video condition unit context adapter