Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive
  Content Generation
This paper explores higher-resolution video outpainting with extensive
content generation. We point out common issues faced by existing methods when
attempting to largely outpaint videos: the generation of low-quality content
and limitations imposed by GPU memory. To address these challenges, we propose
a diffusion-based method called Follow-Your-Canvas. It builds upon two
core designs. First, instead of employing the common practice of "single-shot"
outpainting, we distribute the task across spatial windows and seamlessly merge
them. It allows us to outpaint videos of any size and resolution without being
constrained by GPU memory. Second, the source video and its relative positional
relation are injected into the generation process of each window. It makes the
generated spatial layout within each window harmonize with the source video.
Coupling with these two designs enables us to generate higher-resolution
outpainting videos with rich content while keeping spatial and temporal
consistency. Follow-Your-Canvas excels in large-scale video outpainting, e.g.,
from 512X512 to 1152X2048 (9X), while producing high-quality and aesthetically
pleasing results. It achieves the best quantitative results across various
resolution and scale setups. The code is released on
https://github.com/mayuelala/FollowYourCanvas
66d8050d0aaebff5ea43bcda
Qihua Chen
66d8050d0aaebff5ea43bcdb
Yue Ma
66d8050d0aaebff5ea43bcdc
Hongfa Wang
66d8050d0aaebff5ea43bcdd
Junkun Yuan
66d8050d0aaebff5ea43bcde
Wenzhe Zhao
66d8050d0aaebff5ea43bcdf
Qi Tian
66d8050d0aaebff5ea43bce0
Hongmei Wang
66d8050d0aaebff5ea43bce1
Shaobo Min
66d8050d0aaebff5ea43bce2
Qifeng Chen
admin_assigned
2024-09-04T09:50:21.697Z
6467b121e7a6a374fd19b44b
/avatars/3f2874d58986d651aef55e3408b05700.svg
Qifeng Chen
user
cqf
66d8050d0aaebff5ea43bce3
Wei Liu
Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive
  Content Generation
This paper explores higher-resolution video outpainting with extensive
content generation. We point out common issues faced by existing methods when
attempting to largely outpaint videos: the generation of low-quality content
and limitations imposed by GPU memory. To address these challenges, we propose
a diffusion-based method called Follow-Your-Canvas. It builds upon two
core designs. First, instead of employing the common practice of "single-shot"
outpainting, we distribute the task across spatial windows and seamlessly merge
them. It allows us to outpaint videos of any size and resolution without being
constrained by GPU memory. Second, the source video and its relative positional
relation are injected into the generation process of each window. It makes the
generated spatial layout within each window harmonize with the source video.
Coupling with these two designs enables us to generate higher-resolution
outpainting videos with rich content while keeping spatial and temporal
consistency. Follow-Your-Canvas excels in large-scale video outpainting, e.g.,
from 512X512 to 1152X2048 (9X), while producing high-quality and aesthetically
pleasing results. It achieves the best quantitative results across various
resolution and scale setups. The code is released on
https://github.com/mayuelala/FollowYourCanvas
A diffusion-based method called Follow-Your-Canvas addresses low-quality content generation and GPU memory limitations in video outpainting by distributing the task across spatial windows and considering relative positional relation with the source video.
diffusion-based method Follow-Your-Canvas spatial windows GPU memory source video relative positional relation spatial consistency temporal consistency video outpainting