Trans4D: Realistic Geometry-Aware Transition for Compositional
  Text-to-4D Synthesis
Recent advances in diffusion models have demonstrated exceptional
capabilities in image and video generation, further improving the effectiveness
of 4D synthesis. Existing 4D generation methods can generate high-quality 4D
objects or scenes based on user-friendly conditions, benefiting the gaming and
video industries. However, these methods struggle to synthesize significant
object deformation of complex 4D transitions and interactions within scenes. To
address this challenge, we propose Trans4D, a novel text-to-4D synthesis
framework that enables realistic complex scene transitions. Specifically, we
first use multi-modal large language models (MLLMs) to produce a physic-aware
scene description for 4D scene initialization and effective transition timing
planning. Then we propose a geometry-aware 4D transition network to realize a
complex scene-level 4D transition based on the plan, which involves expressive
geometrical object deformation. Extensive experiments demonstrate that Trans4D
consistently outperforms existing state-of-the-art methods in generating 4D
scenes with accurate and high-quality transitions, validating its
effectiveness. Code: https://github.com/YangLing0818/Trans4D
67074ed5b20a8ecb7b6c0826
Bohan Zeng
claimed_verified
2024-10-10T08:00:40.793Z
6671214c92412fd4640714eb
/avatars/48fa84e7bc3bb92ad0192aa26b32de10.svg
bohan zeng
user
zbhpku
67074ed5b20a8ecb7b6c0827
Ling Yang
claimed_verified
2025-02-11T15:34:25.514Z
64fde4e252e82dd432b74ce9
https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg
Ling Yang
user
Lingaaaaaaa
67074ed5b20a8ecb7b6c0828
Siyu Li
67074ed5b20a8ecb7b6c0829
Jiaming Liu
claimed_verified
2025-01-31T08:37:26.855Z
637c941588699fba70e29f70
https://cdn-avatars.huggingface.co/v1/production/uploads/637c941588699fba70e29f70/b6G_QZkT-MhE47dx87i0d.png
LIU JIAMING
user
jamesliu1217
67074ed5b20a8ecb7b6c082a
Zixiang Zhang
67074ed5b20a8ecb7b6c082b
Juanxi Tian
claimed_verified
2024-10-11T07:44:12.403Z
670880950e79a8b46f7ff9dd
https://cdn-avatars.huggingface.co/v1/production/uploads/670880950e79a8b46f7ff9dd/hA1TLhwlQblkFsq8wLrkB.jpeg
Juanxi Tian
user
Juanxi
67074ed5b20a8ecb7b6c082c
Kaixin Zhu
claimed_verified
2025-03-18T09:16:50.446Z
6708920aeae29d1cd41a703b
/avatars/922427a86523b0aa810412fd2d75f88e.svg
kaixin zhu
user
czkk566
67074ed5b20a8ecb7b6c082d
Yongzhen Guo
67074ed5b20a8ecb7b6c082e
Fu-Yun Wang
claimed_verified
2024-10-13T20:18:04.837Z
63e9e92f20c109718713f5eb
/avatars/9ff312e854d803e1a2e9e685a21d12f8.svg
Fu-Yun Wang
user
wangfuyun
67074ed5b20a8ecb7b6c082f
Minkai Xu
67074ed5b20a8ecb7b6c0830
Stefano Ermon
67074ed5b20a8ecb7b6c0831
Wentao Zhang
Trans4D: Realistic Geometry-Aware Transition for Compositional
  Text-to-4D Synthesis
Recent advances in diffusion models have demonstrated exceptional
capabilities in image and video generation, further improving the effectiveness
of 4D synthesis. Existing 4D generation methods can generate high-quality 4D
objects or scenes based on user-friendly conditions, benefiting the gaming and
video industries. However, these methods struggle to synthesize significant
object deformation of complex 4D transitions and interactions within scenes. To
address this challenge, we propose Trans4D, a novel text-to-4D synthesis
framework that enables realistic complex scene transitions. Specifically, we
first use multi-modal large language models (MLLMs) to produce a physic-aware
scene description for 4D scene initialization and effective transition timing
planning. Then we propose a geometry-aware 4D transition network to realize a
complex scene-level 4D transition based on the plan, which involves expressive
geometrical object deformation. Extensive experiments demonstrate that Trans4D
consistently outperforms existing state-of-the-art methods in generating 4D
scenes with accurate and high-quality transitions, validating its
effectiveness. Code: https://github.com/YangLing0818/Trans4D
Trans4D generates realistic complex scene transitions in 4D synthesis using multi-modal large language models and a geometry-aware transition network.
diffusion models 4D synthesis text-to-4D synthesis multi-modal large language models geometry-aware transition network scene transitions object deformation