title: VIA: A Spatiotemporal Video Adaptation Framework for Global and Local
    Video Editing
summary: Video editing stands as a cornerstone of digital media, from entertainment
  and education to professional communication. However, previous methods often
  overlook the necessity of comprehensively understanding both global and local
  contexts, leading to inaccurate and inconsistency edits in the spatiotemporal
  dimension, especially for long videos. In this paper, we introduce VIA, a
  unified spatiotemporal VIdeo Adaptation framework for global and local video
  editing, pushing the limits of consistently editing minute-long videos. First,
  to ensure local consistency within individual frames, the foundation of VIA is
  a novel test-time editing adaptation method, which adapts a pre-trained image
  editing model for improving consistency between potential editing directions
  and the text instruction, and adapts masked latent variables for precise local
  control. Furthermore, to maintain global consistency over the video sequence,
  we introduce spatiotemporal adaptation that adapts consistent attention
  variables in key frames and strategically applies them across the whole
  sequence to realize the editing effects. Extensive experiments demonstrate
  that, compared to baseline methods, our VIA approach produces edits that are
  more faithful to the source videos, more coherent in the spatiotemporal
  context, and more precise in local control. More importantly, we show that VIA
  can achieve consistent long video editing in minutes, unlocking the potentials
  for advanced video editing tasks over long video sequences.
paper_authors._id: 66731088881ffb63ca260483
paper_authors.name: Jing Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-20T09:37:09.202Z
paper_authors._id: 634781fae5c0717e6739bc59
paper_authors.avatarUrl: /avatars/2b6d2d0625741b28f326459acad55b69.svg
paper_authors.fullname: Jing Gu
paper_authors.type: user
paper_authors.user: jinggu
paper_authors._id: 66731088881ffb63ca260484
paper_authors.name: Yuwei Fang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-20T09:37:42.492Z
paper_authors._id: 6406d1dcc3ab325efa9db198
paper_authors.avatarUrl: /avatars/781f3cd455596f7cffa0906cc487f79b.svg
paper_authors.fullname: Yuwei Fang
paper_authors.type: user
paper_authors.user: studyfang
paper_authors._id: 66731088881ffb63ca260485
paper_authors.name: Ivan Skorokhodov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-20T09:37:52.168Z
paper_authors._id: 63610db13c7147fae7de88e3
paper_authors.avatarUrl: /avatars/d7e97a16cfee39e1e50d7a5b747876f1.svg
paper_authors.fullname: Ivan Skorokhodov
paper_authors.type: user
paper_authors.user: universome
paper_authors._id: 66731088881ffb63ca260486
paper_authors.name: Peter Wonka
paper_authors._id: 66731088881ffb63ca260487
paper_authors.name: Xinya Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-16T07:03:31.181Z
paper_authors._id: 66c4154fb83a7e94d588ade3
paper_authors.avatarUrl: /avatars/2f1c8a3feb81be628e36eaa198450704.svg
paper_authors.fullname: Xinya Du
paper_authors.type: user
paper_authors.user: xinyadu
paper_authors._id: 66731088881ffb63ca260488
paper_authors.name: Sergey Tulyakov
paper_authors._id: 66731088881ffb63ca260489
paper_authors.name: Xin Eric Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-20T07:34:17.594Z
paper_authors._id: 64679a226192d39142245e5e
paper_authors.avatarUrl: /avatars/05abee0b6317f100923936ca2099e9eb.svg
paper_authors.fullname: Xin Eric Wang
paper_authors.type: user
paper_authors.user: xw-eric
paper_title: VIA: A Spatiotemporal Video Adaptation Framework for Global and Local
    Video Editing
paper_summary: Video editing stands as a cornerstone of digital media, from entertainment
  and education to professional communication. However, previous methods often
  overlook the necessity of comprehensively understanding both global and local
  contexts, leading to inaccurate and inconsistency edits in the spatiotemporal
  dimension, especially for long videos. In this paper, we introduce VIA, a
  unified spatiotemporal VIdeo Adaptation framework for global and local video
  editing, pushing the limits of consistently editing minute-long videos. First,
  to ensure local consistency within individual frames, the foundation of VIA is
  a novel test-time editing adaptation method, which adapts a pre-trained image
  editing model for improving consistency between potential editing directions
  and the text instruction, and adapts masked latent variables for precise local
  control. Furthermore, to maintain global consistency over the video sequence,
  we introduce spatiotemporal adaptation that adapts consistent attention
  variables in key frames and strategically applies them across the whole
  sequence to realize the editing effects. Extensive experiments demonstrate
  that, compared to baseline methods, our VIA approach produces edits that are
  more faithful to the source videos, more coherent in the spatiotemporal
  context, and more precise in local control. More importantly, we show that VIA
  can achieve consistent long video editing in minutes, unlocking the potentials
  for advanced video editing tasks over long video sequences.
paper_ai_summary: VIA, a unified spatiotemporal video editing framework, enhances consistency and precision in global and local editing of long videos.
paper_ai_keywords: spatiotemporal adaptation test-time editing adaptation pre-trained image editing model masked latent variables consistent attention variables vide editing spatiotemporal context local control global consistency