PlayerOne: Egocentric World Simulator
We introduce PlayerOne, the first egocentric realistic world simulator,
facilitating immersive and unrestricted exploration within vividly dynamic
environments. Given an egocentric scene image from the user, PlayerOne can
accurately construct the corresponding world and generate egocentric videos
that are strictly aligned with the real scene human motion of the user captured
by an exocentric camera. PlayerOne is trained in a coarse-to-fine pipeline that
first performs pretraining on large-scale egocentric text-video pairs for
coarse-level egocentric understanding, followed by finetuning on synchronous
motion-video data extracted from egocentric-exocentric video datasets with our
automatic construction pipeline. Besides, considering the varying importance of
different components, we design a part-disentangled motion injection scheme,
enabling precise control of part-level movements. In addition, we devise a
joint reconstruction framework that progressively models both the 4D scene and
video frames, ensuring scene consistency in the long-form video generation.
Experimental results demonstrate its great generalization ability in precise
control of varying human movements and worldconsistent modeling of diverse
scenarios. It marks the first endeavor into egocentric real-world simulation
and can pave the way for the community to delve into fresh frontiers of world
modeling and its diverse applications.
684a39639b38e1e5a33a6837
Yuanpeng Tu
684a39639b38e1e5a33a6838
Hao Luo
684a39639b38e1e5a33a6839
Xi Chen
claimed_verified
2025-06-12T12:42:07.608Z
644a1b6401e18bf93a6f45c1
https://cdn-avatars.huggingface.co/v1/production/uploads/644a1b6401e18bf93a6f45c1/P0i_CgCrIzOS2tYRlxoE9.png
xichen
user
xichenhku
684a39639b38e1e5a33a683a
Xiang Bai
684a39639b38e1e5a33a683b
Fan Wang
684a39639b38e1e5a33a683c
Hengshuang Zhao
PlayerOne: Egocentric World Simulator
We introduce PlayerOne, the first egocentric realistic world simulator,
facilitating immersive and unrestricted exploration within vividly dynamic
environments. Given an egocentric scene image from the user, PlayerOne can
accurately construct the corresponding world and generate egocentric videos
that are strictly aligned with the real scene human motion of the user captured
by an exocentric camera. PlayerOne is trained in a coarse-to-fine pipeline that
first performs pretraining on large-scale egocentric text-video pairs for
coarse-level egocentric understanding, followed by finetuning on synchronous
motion-video data extracted from egocentric-exocentric video datasets with our
automatic construction pipeline. Besides, considering the varying importance of
different components, we design a part-disentangled motion injection scheme,
enabling precise control of part-level movements. In addition, we devise a
joint reconstruction framework that progressively models both the 4D scene and
video frames, ensuring scene consistency in the long-form video generation.
Experimental results demonstrate its great generalization ability in precise
control of varying human movements and worldconsistent modeling of diverse
scenarios. It marks the first endeavor into egocentric real-world simulation
and can pave the way for the community to delve into fresh frontiers of world
modeling and its diverse applications.
PlayerOne is an egocentric realistic world simulator that constructs and generates videos from user-captured images, using a coarse-to-fine training pipeline and advanced motion injection and reconstruction frameworks.
egocentric realistic world simulator coarse-to-fine pipeline pretraining finetuning synchronous motion-video data automatic construction pipeline part-disentangled motion injection joint reconstruction framework 4D scene video frames scene consistency long-form video generation worldconsistent modeling