Matrix-Game 2.0: An Open-Source, Real-Time, and Streaming Interactive World Model

📝 Overview

Matrix-Game-2.0（1.8B） is an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion

✨ Key Features

🚀 Feature 1: Real-Time Distillation Efficient few-step diffusion for streaming video synthesis at 25 FPS, producing minute-level, high-fidelity videos across complex environments with ultra-fast speed.
🖱️ Feature 2: Precise Action Injection A mouse/keyboard-to-frame module that embeds user inputs as direct interactions, enabling frame-level control and dynamic response in generated videos.
🎬 Feature 3: Massive Interactive Data Pipeline A scalable production system for Unreal Engine & GTA5 that generates ~1200 hours of high-quality interactive video data, covering diverse scenes with frame-level realism.

🔥 Latest Updates

[2025-08] 🎉 Initial release of Matrix-Game-2.0 Model

Model Overview

Matrix-Game-2.0（1.8B） is derived from the Wan. By removing the text branch and adding action modules, the model predicts next frames only from visual contents and corresponding actions.

📈 Performance Comparison

GameWorld Score Benchmark Comparison

Model	Image Quality ↑	Aesthetic Quality ↑	Temporal Cons. ↑	Motion Smooth. ↑	Keyboard Acc. ↑	Mouse Acc. ↑	Object Cons.	Scenario Cons.
Oasis	0.27	0.27	0.82	0.99	0.73	0.56	0.18	0.84
Ours	0.61	0.50	0.94	0.98	0.91	0.95	0.64	0.80

Metric Descriptions:

Image Quality / Aesthetic: Visual fidelity and perceptual appeal of generated frames
Temporal Consistency / Motion Smoothness: Temporal coherence and smoothness between frames
Keyboard Accuracy / Mouse Accuracy: Accuracy in following user control signals
Object Consistency: Geometric stability and consistency of objects over time
Scenario Consistency: Scenario consistency over time

Please check our GameWorld benchmark for detailed implementation.

🚀 Quick Start

# clone the repository:
git clone https://github.com/SkyworkAI/Matrix-Game.git
cd Matrix-Game/Matrix-Game-2

# install apex and FlashAttention
# Our project also depends on [FlashAttention](https://github.com/Dao-AILab/flash-attention)
# install dependencies:
pip install -r requirements.txt
python setup.py develop

# inference
python inference.py \
    --config_path configs/inference_yaml/{your-config}.yaml \
    --checkpoint_path {path-to-the-checkpoint} \
    --img_path {path-to-the-input-image} \
    --output_folder outputs \
    --num_output_frames 150 \
    --seed 42 \
    --pretrained_model_path {path-to-the-vae-folder}
# inference streaming
python inference_streaming.py \
    --config_path configs/inference_yaml/{your-config}.yaml \
    --checkpoint_path {path-to-the-checkpoint} \
    --output_folder outputs \
    --seed 42 \
    --pretrained_model_path {path-to-the-vae-folder}

⭐ Acknowledgements

We would like to express our gratitude to:

Diffusers for their excellent diffusion model framework
SkyReels-V2 for their strong base model
Self-Forcing for their excellent work
MineRL for their excellent gym framework
Video-Pre-Training for their accurate Inverse Dynamics Model
GameFactory for their idea of action control module

We are grateful to the broader research community for their open exploration and contributions to the field of interactive world generation.

📎 Citation

If you find this project useful, please cite our paper:

Skywork
/

Matrix-Game-2.0