Matrix-Game: Interactive World Foundation Model
We introduce Matrix-Game, an interactive world foundation model for
controllable game world generation. Matrix-Game is trained using a two-stage
pipeline that first performs large-scale unlabeled pretraining for environment
understanding, followed by action-labeled training for interactive video
generation. To support this, we curate Matrix-Game-MC, a comprehensive
Minecraft dataset comprising over 2,700 hours of unlabeled gameplay video clips
and over 1,000 hours of high-quality labeled clips with fine-grained keyboard
and mouse action annotations. Our model adopts a controllable image-to-world
generation paradigm, conditioned on a reference image, motion context, and user
actions. With over 17 billion parameters, Matrix-Game enables precise control
over character actions and camera movements, while maintaining high visual
quality and temporal coherence. To evaluate performance, we develop GameWorld
Score, a unified benchmark measuring visual quality, temporal quality, action
controllability, and physical rule understanding for Minecraft world
generation. Extensive experiments show that Matrix-Game consistently
outperforms prior open-source Minecraft world models (including Oasis and
MineWorld) across all metrics, with particularly strong gains in
controllability and physical consistency. Double-blind human evaluations
further confirm the superiority of Matrix-Game, highlighting its ability to
generate perceptually realistic and precisely controllable videos across
diverse game scenarios. To facilitate future research on interactive
image-to-world generation, we will open-source the Matrix-Game model weights
and the GameWorld Score benchmark at https://github.com/SkyworkAI/Matrix-Game.
685a14da0e4ad7e21975854d
Yifan Zhang
claimed_verified
2025-06-25T08:09:21.431Z
63aed0e7f873109b112dbb1b
https://cdn-avatars.huggingface.co/v1/production/uploads/63aed0e7f873109b112dbb1b/JkSkQ1a2SLq5eTTnz9F05.jpeg
Yifan Zhang
user
Vanint
685a14da0e4ad7e21975854e
Chunli Peng
admin_assigned
2025-06-25T13:32:40.141Z
68378e584ed6982099b1a1aa
https://cdn-avatars.huggingface.co/v1/production/uploads/68378e584ed6982099b1a1aa/B23K1b3EoRYPDTbYYbPzY.jpeg
CHUNLI PENG
user
chunli-peng
685a14da0e4ad7e21975854f
Boyang Wang
claimed_verified
2025-07-03T07:41:41.368Z
663f30673640ac64c4fcf9a4
https://cdn-avatars.huggingface.co/v1/production/uploads/663f30673640ac64c4fcf9a4/xCtO-Mt-3f4MlzjSwM3Uf.jpeg
Boyang Wang
user
brycebywang
685a14da0e4ad7e219758550
Puyi Wang
admin_assigned
2025-06-25T13:33:20.411Z
664717a50860c78e7c7b7c52
/avatars/ca17216b6d73234e1a68510f87653b3a.svg
Puyi Wang
user
Puyiiii
685a14da0e4ad7e219758551
Qingcheng Zhu
685a14da0e4ad7e219758552
Fei Kang
685a14da0e4ad7e219758553
Biao Jiang
685a14da0e4ad7e219758554
Zedong Gao
685a14da0e4ad7e219758555
Eric Li
685a14da0e4ad7e219758556
Yang Liu
685a14da0e4ad7e219758557
Yahui Zhou
Matrix-Game: Interactive World Foundation Model
We introduce Matrix-Game, an interactive world foundation model for
controllable game world generation. Matrix-Game is trained using a two-stage
pipeline that first performs large-scale unlabeled pretraining for environment
understanding, followed by action-labeled training for interactive video
generation. To support this, we curate Matrix-Game-MC, a comprehensive
Minecraft dataset comprising over 2,700 hours of unlabeled gameplay video clips
and over 1,000 hours of high-quality labeled clips with fine-grained keyboard
and mouse action annotations. Our model adopts a controllable image-to-world
generation paradigm, conditioned on a reference image, motion context, and user
actions. With over 17 billion parameters, Matrix-Game enables precise control
over character actions and camera movements, while maintaining high visual
quality and temporal coherence. To evaluate performance, we develop GameWorld
Score, a unified benchmark measuring visual quality, temporal quality, action
controllability, and physical rule understanding for Minecraft world
generation. Extensive experiments show that Matrix-Game consistently
outperforms prior open-source Minecraft world models (including Oasis and
MineWorld) across all metrics, with particularly strong gains in
controllability and physical consistency. Double-blind human evaluations
further confirm the superiority of Matrix-Game, highlighting its ability to
generate perceptually realistic and precisely controllable videos across
diverse game scenarios. To facilitate future research on interactive
image-to-world generation, we will open-source the Matrix-Game model weights
and the GameWorld Score benchmark at https://github.com/SkyworkAI/Matrix-Game.
Matrix-Game, a controllable game world generation model trained in a two-stage process, outperforms existing models by producing high-quality, action-controllable, and physically consistent Minecraft world videos.
Matrix-Game interactive world foundation model large-scale unlabeled pretraining action-labeled training contrrollable image-to-world generation Matrix-Game-MC motion context character actions camera movements visual quality temporal coherence GameWorld Score double-blind human evaluations interactive image-to-world generation Oasis MineWorld perceptually realistic