Imagine360: Immersive 360 Video Generation from Perspective Anchor
360^circ videos offer a hyper-immersive experience that allows the viewers
to explore a dynamic scene from full 360 degrees. To achieve more user-friendly
and personalized content creation in 360^circ video format, we seek to lift
standard perspective videos into 360^circ equirectangular videos. To this
end, we introduce Imagine360, the first perspective-to-360^circ video
generation framework that creates high-quality 360^circ videos with rich and
diverse motion patterns from video anchors. Imagine360 learns fine-grained
spherical visual and motion patterns from limited 360^circ video data with
several key designs. 1) Firstly we adopt the dual-branch design, including a
perspective and a panorama video denoising branch to provide local and global
constraints for 360^circ video generation, with motion module and spatial
LoRA layers fine-tuned on extended web 360^circ videos. 2) Additionally, an
antipodal mask is devised to capture long-range motion dependencies, enhancing
the reversed camera motion between antipodal pixels across hemispheres. 3) To
handle diverse perspective video inputs, we propose elevation-aware designs
that adapt to varying video masking due to changing elevations across frames.
Extensive experiments show Imagine360 achieves superior graphics quality and
motion coherence among state-of-the-art 360^circ video generation methods.
We believe Imagine360 holds promise for advancing personalized, immersive
360^circ video creation.
675118447679a2657e4d1369
Jing Tan
admin_assigned
2024-12-05T09:33:59.672Z
65367c40061949598892dbdc
/avatars/4baf27263841471cbd5f629a8b99424d.svg
Jing Tan
user
jingtan
675118447679a2657e4d136a
Shuai Yang
claimed_verified
2025-06-06T07:42:58.099Z
6373037cd3ba6bd3f9bc32fa
https://cdn-avatars.huggingface.co/v1/production/uploads/6373037cd3ba6bd3f9bc32fa/dIL4AIcL24QL_Rp4B9ewS.png
Shuai Yang
user
ysmikey
675118447679a2657e4d136b
Tong Wu
admin_assigned
2024-12-05T09:33:26.828Z
62fae9328e137d7c4b896498
/avatars/1bda39dec585c099417cc9daa9f53c42.svg
Tong Wu
user
tongwu2020
675118447679a2657e4d136c
Jingwen He
admin_assigned
2024-12-05T09:33:19.535Z
670749a9d827da9f37508209
/avatars/f14fc05ad405f3967b9af0bcc73d4207.svg
he jingwen
user
mimihe
675118447679a2657e4d136d
Yuwei Guo
admin_assigned
2024-12-05T09:33:06.811Z
6371f83d5ffa922d638ef486
https://cdn-avatars.huggingface.co/v1/production/uploads/6371f83d5ffa922d638ef486/UzWl3xhVvtol3Rz1mjQ9t.jpeg
Yuwei Guo
user
guoyww
675118447679a2657e4d136e
Ziwei Liu
admin_assigned
2024-12-05T09:32:44.270Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
675118447679a2657e4d136f
Dahua Lin
admin_assigned
2024-12-05T09:32:38.520Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
Imagine360: Immersive 360 Video Generation from Perspective Anchor
360^circ videos offer a hyper-immersive experience that allows the viewers
to explore a dynamic scene from full 360 degrees. To achieve more user-friendly
and personalized content creation in 360^circ video format, we seek to lift
standard perspective videos into 360^circ equirectangular videos. To this
end, we introduce Imagine360, the first perspective-to-360^circ video
generation framework that creates high-quality 360^circ videos with rich and
diverse motion patterns from video anchors. Imagine360 learns fine-grained
spherical visual and motion patterns from limited 360^circ video data with
several key designs. 1) Firstly we adopt the dual-branch design, including a
perspective and a panorama video denoising branch to provide local and global
constraints for 360^circ video generation, with motion module and spatial
LoRA layers fine-tuned on extended web 360^circ videos. 2) Additionally, an
antipodal mask is devised to capture long-range motion dependencies, enhancing
the reversed camera motion between antipodal pixels across hemispheres. 3) To
handle diverse perspective video inputs, we propose elevation-aware designs
that adapt to varying video masking due to changing elevations across frames.
Extensive experiments show Imagine360 achieves superior graphics quality and
motion coherence among state-of-the-art 360^circ video generation methods.
We believe Imagine360 holds promise for advancing personalized, immersive
360^circ video creation.
Imagine360 generates high-quality, personalized 360° videos with rich motion patterns using dual-branch denoising, antipodal masks, and elevation-aware designs.
dual-branch design perspective video denoising panorama video denoising motion module spatial LoRA layers antipodal mask elevation-aware designs