RoboScape: Physics-informed Embodied World Model
World models have become indispensable tools for embodied intelligence,
serving as powerful simulators capable of generating realistic robotic videos
while addressing critical data scarcity challenges. However, current embodied
world models exhibit limited physical awareness, particularly in modeling 3D
geometry and motion dynamics, resulting in unrealistic video generation for
contact-rich robotic scenarios. In this paper, we present RoboScape, a unified
physics-informed world model that jointly learns RGB video generation and
physics knowledge within an integrated framework. We introduce two key
physics-informed joint training tasks: temporal depth prediction that enhances
3D geometric consistency in video rendering, and keypoint dynamics learning
that implicitly encodes physical properties (e.g., object shape and material
characteristics) while improving complex motion modeling. Extensive experiments
demonstrate that RoboScape generates videos with superior visual fidelity and
physical plausibility across diverse robotic scenarios. We further validate its
practical utility through downstream applications including robotic policy
training with generated data and policy evaluation. Our work provides new
insights for building efficient physics-informed world models to advance
embodied intelligence research. The code is available at:
https://github.com/tsinghua-fib-lab/RoboScape.
68638235588cea0da970c966
Yu Shang
68638235588cea0da970c967
Xin Zhang
68638235588cea0da970c968
Yinzhou Tang
68638235588cea0da970c969
Lei Jin
68638235588cea0da970c96a
Chen Gao
68638235588cea0da970c96b
Wei Wu
68638235588cea0da970c96c
Yong Li
RoboScape: Physics-informed Embodied World Model
World models have become indispensable tools for embodied intelligence,
serving as powerful simulators capable of generating realistic robotic videos
while addressing critical data scarcity challenges. However, current embodied
world models exhibit limited physical awareness, particularly in modeling 3D
geometry and motion dynamics, resulting in unrealistic video generation for
contact-rich robotic scenarios. In this paper, we present RoboScape, a unified
physics-informed world model that jointly learns RGB video generation and
physics knowledge within an integrated framework. We introduce two key
physics-informed joint training tasks: temporal depth prediction that enhances
3D geometric consistency in video rendering, and keypoint dynamics learning
that implicitly encodes physical properties (e.g., object shape and material
characteristics) while improving complex motion modeling. Extensive experiments
demonstrate that RoboScape generates videos with superior visual fidelity and
physical plausibility across diverse robotic scenarios. We further validate its
practical utility through downstream applications including robotic policy
training with generated data and policy evaluation. Our work provides new
insights for building efficient physics-informed world models to advance
embodied intelligence research. The code is available at:
https://github.com/tsinghua-fib-lab/RoboScape.
RoboScape is a physics-informed world model that integrates RGB video generation and physics knowledge, improving visual fidelity and physical plausibility in robotic scenarios.
world models embodied intelligence RGB video generation physics knowledge temporal depth prediction keypoint dynamics learning visual fidelity physical plausibility robotic policy training policy evaluation