title: Trans4D: Realistic Geometry-Aware Transition for Compositional
    Text-to-4D Synthesis
summary: Recent advances in diffusion models have demonstrated exceptional
  capabilities in image and video generation, further improving the effectiveness
  of 4D synthesis. Existing 4D generation methods can generate high-quality 4D
  objects or scenes based on user-friendly conditions, benefiting the gaming and
  video industries. However, these methods struggle to synthesize significant
  object deformation of complex 4D transitions and interactions within scenes. To
  address this challenge, we propose Trans4D, a novel text-to-4D synthesis
  framework that enables realistic complex scene transitions. Specifically, we
  first use multi-modal large language models (MLLMs) to produce a physic-aware
  scene description for 4D scene initialization and effective transition timing
  planning. Then we propose a geometry-aware 4D transition network to realize a
  complex scene-level 4D transition based on the plan, which involves expressive
  geometrical object deformation. Extensive experiments demonstrate that Trans4D
  consistently outperforms existing state-of-the-art methods in generating 4D
  scenes with accurate and high-quality transitions, validating its
  effectiveness. Code: https://github.com/YangLing0818/Trans4D
paper_authors._id: 67074ed5b20a8ecb7b6c0826
paper_authors.name: Bohan Zeng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T08:00:40.793Z
paper_authors._id: 6671214c92412fd4640714eb
paper_authors.avatarUrl: /avatars/48fa84e7bc3bb92ad0192aa26b32de10.svg
paper_authors.fullname: bohan zeng
paper_authors.type: user
paper_authors.user: zbhpku
paper_authors._id: 67074ed5b20a8ecb7b6c0827
paper_authors.name: Ling Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-11T15:34:25.514Z
paper_authors._id: 64fde4e252e82dd432b74ce9
paper_authors.avatarUrl: /avatars/061a69d858b86d1600be916122cae7fc.svg
paper_authors.fullname: Ling Yang
paper_authors.type: user
paper_authors.user: Lingaaaaaaa
paper_authors._id: 67074ed5b20a8ecb7b6c0828
paper_authors.name: Siyu Li
paper_authors._id: 67074ed5b20a8ecb7b6c0829
paper_authors.name: Jiaming Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-31T08:37:26.855Z
paper_authors._id: 637c941588699fba70e29f70
paper_authors.avatarUrl: /avatars/1b6ce2faf0f8c98b987d8efc9b92cc31.svg
paper_authors.fullname: LIU JIAMING
paper_authors.type: user
paper_authors.user: jamesliu1217
paper_authors._id: 67074ed5b20a8ecb7b6c082a
paper_authors.name: Zixiang Zhang
paper_authors._id: 67074ed5b20a8ecb7b6c082b
paper_authors.name: Juanxi Tian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-11T07:44:12.403Z
paper_authors._id: 670880950e79a8b46f7ff9dd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/670880950e79a8b46f7ff9dd/hA1TLhwlQblkFsq8wLrkB.jpeg
paper_authors.fullname: Juanxi Tian
paper_authors.type: user
paper_authors.user: Juanxi
paper_authors._id: 67074ed5b20a8ecb7b6c082c
paper_authors.name: Kaixin Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-18T09:16:50.446Z
paper_authors._id: 6708920aeae29d1cd41a703b
paper_authors.avatarUrl: /avatars/922427a86523b0aa810412fd2d75f88e.svg
paper_authors.fullname: kaixin zhu
paper_authors.type: user
paper_authors.user: czkk566
paper_authors._id: 67074ed5b20a8ecb7b6c082d
paper_authors.name: Yongzhen Guo
paper_authors._id: 67074ed5b20a8ecb7b6c082e
paper_authors.name: Fu-Yun Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-13T20:18:04.837Z
paper_authors._id: 63e9e92f20c109718713f5eb
paper_authors.avatarUrl: /avatars/9ff312e854d803e1a2e9e685a21d12f8.svg
paper_authors.fullname: Fu-Yun Wang
paper_authors.type: user
paper_authors.user: wangfuyun
paper_authors._id: 67074ed5b20a8ecb7b6c082f
paper_authors.name: Minkai Xu
paper_authors._id: 67074ed5b20a8ecb7b6c0830
paper_authors.name: Stefano Ermon
paper_authors._id: 67074ed5b20a8ecb7b6c0831
paper_authors.name: Wentao Zhang
paper_title: Trans4D: Realistic Geometry-Aware Transition for Compositional
    Text-to-4D Synthesis
paper_summary: Recent advances in diffusion models have demonstrated exceptional
  capabilities in image and video generation, further improving the effectiveness
  of 4D synthesis. Existing 4D generation methods can generate high-quality 4D
  objects or scenes based on user-friendly conditions, benefiting the gaming and
  video industries. However, these methods struggle to synthesize significant
  object deformation of complex 4D transitions and interactions within scenes. To
  address this challenge, we propose Trans4D, a novel text-to-4D synthesis
  framework that enables realistic complex scene transitions. Specifically, we
  first use multi-modal large language models (MLLMs) to produce a physic-aware
  scene description for 4D scene initialization and effective transition timing
  planning. Then we propose a geometry-aware 4D transition network to realize a
  complex scene-level 4D transition based on the plan, which involves expressive
  geometrical object deformation. Extensive experiments demonstrate that Trans4D
  consistently outperforms existing state-of-the-art methods in generating 4D
  scenes with accurate and high-quality transitions, validating its
  effectiveness. Code: https://github.com/YangLing0818/Trans4D
paper_ai_keywords: diffusion models 4D synthesis text-to-4D synthesis multi-modal large language models (MLLMs) physics-aware scene description geometry-aware 4D transition network complex scene-level 4D transition expressive geometrical object deformation