CityDreamer4D: Compositional Generative Model of Unbounded 4D Cities
3D scene generation has garnered growing attention in recent years and has
made significant progress. Generating 4D cities is more challenging than 3D
scenes due to the presence of structurally complex, visually diverse objects
like buildings and vehicles, and heightened human sensitivity to distortions in
urban environments. To tackle these issues, we propose CityDreamer4D, a
compositional generative model specifically tailored for generating unbounded
4D cities. Our main insights are 1) 4D city generation should separate dynamic
objects (e.g., vehicles) from static scenes (e.g., buildings and roads), and 2)
all objects in the 4D scene should be composed of different types of neural
fields for buildings, vehicles, and background stuff. Specifically, we propose
Traffic Scenario Generator and Unbounded Layout Generator to produce dynamic
traffic scenarios and static city layouts using a highly compact BEV
representation. Objects in 4D cities are generated by combining stuff-oriented
and instance-oriented neural fields for background stuff, buildings, and
vehicles. To suit the distinct characteristics of background stuff and
instances, the neural fields employ customized generative hash grids and
periodic positional embeddings as scene parameterizations. Furthermore, we
offer a comprehensive suite of datasets for city generation, including OSM,
GoogleEarth, and CityTopia. The OSM dataset provides a variety of real-world
city layouts, while the Google Earth and CityTopia datasets deliver
large-scale, high-quality city imagery complete with 3D instance annotations.
Leveraging its compositional design, CityDreamer4D supports a range of
downstream applications, such as instance editing, city stylization, and urban
simulation, while delivering state-of-the-art performance in generating
realistic 4D cities.
678897a4d42825b51c19d65a
Haozhe Xie
admin_assigned
2025-01-16T08:50:10.101Z
63f47b5321eb234ab739e91a
https://cdn-avatars.huggingface.co/v1/production/uploads/63f47b5321eb234ab739e91a/vWfFNVtMkHl8gieha5PPd.jpeg
Haozhe Xie
user
hzxie
678897a4d42825b51c19d65b
Zhaoxi Chen
admin_assigned
2025-01-16T08:50:16.000Z
62fc8cf7ee999004b5a8b982
/avatars/6c5dda9e58747054a989f077a078f3dc.svg
Zhaoxi Chen
user
FrozenBurning
678897a4d42825b51c19d65c
Fangzhou Hong
admin_assigned
2025-01-16T08:51:11.083Z
623c530013a63ea865f96c8e
/avatars/164455a1a94f92b71733fc778c21bd89.svg
Fangzhou Hong
user
hongfz16
678897a4d42825b51c19d65d
Ziwei Liu
admin_assigned
2025-01-16T08:50:34.647Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
CityDreamer4D: Compositional Generative Model of Unbounded 4D Cities
3D scene generation has garnered growing attention in recent years and has
made significant progress. Generating 4D cities is more challenging than 3D
scenes due to the presence of structurally complex, visually diverse objects
like buildings and vehicles, and heightened human sensitivity to distortions in
urban environments. To tackle these issues, we propose CityDreamer4D, a
compositional generative model specifically tailored for generating unbounded
4D cities. Our main insights are 1) 4D city generation should separate dynamic
objects (e.g., vehicles) from static scenes (e.g., buildings and roads), and 2)
all objects in the 4D scene should be composed of different types of neural
fields for buildings, vehicles, and background stuff. Specifically, we propose
Traffic Scenario Generator and Unbounded Layout Generator to produce dynamic
traffic scenarios and static city layouts using a highly compact BEV
representation. Objects in 4D cities are generated by combining stuff-oriented
and instance-oriented neural fields for background stuff, buildings, and
vehicles. To suit the distinct characteristics of background stuff and
instances, the neural fields employ customized generative hash grids and
periodic positional embeddings as scene parameterizations. Furthermore, we
offer a comprehensive suite of datasets for city generation, including OSM,
GoogleEarth, and CityTopia. The OSM dataset provides a variety of real-world
city layouts, while the Google Earth and CityTopia datasets deliver
large-scale, high-quality city imagery complete with 3D instance annotations.
Leveraging its compositional design, CityDreamer4D supports a range of
downstream applications, such as instance editing, city stylization, and urban
simulation, while delivering state-of-the-art performance in generating
realistic 4D cities.
CityDreamer4D generates realistic 4D cities by separating dynamic and static elements using customized neural fields and compositional generative models.
compositional generative model Traffic Scenario Generator Unbounded Layout Generator BEV representation neural fields generative hash grids periodic positional embeddings scene parameterizations