title: GameFactory: Creating New Games with Generative Interactive Videos
summary: Generative game engines have the potential to revolutionize game development
  by autonomously creating new content and reducing manual workload. However,
  existing video-based game generation methods fail to address the critical
  challenge of scene generalization, limiting their applicability to existing
  games with fixed styles and scenes. In this paper, we present GameFactory, a
  framework focused on exploring scene generalization in game video generation.
  To enable the creation of entirely new and diverse games, we leverage
  pre-trained video diffusion models trained on open-domain video data. To bridge
  the domain gap between open-domain priors and small-scale game dataset, we
  propose a multi-phase training strategy that decouples game style learning from
  action control, preserving open-domain generalization while achieving action
  controllability. Using Minecraft as our data source, we release GF-Minecraft, a
  high-quality and diversity action-annotated video dataset for research.
  Furthermore, we extend our framework to enable autoregressive
  action-controllable game video generation, allowing the production of
  unlimited-length interactive game videos. Experimental results demonstrate that
  GameFactory effectively generates open-domain, diverse, and action-controllable
  game videos, representing a significant step forward in AI-driven game
  generation. Our dataset and project page are publicly available at
  https://vvictoryuki.github.io/gamefactory/.
paper_authors._id: 678719ac5333dfbf8e206077
paper_authors.name: Jiwen Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-15T08:49:01.033Z
paper_authors._id: 64105a6d14215c0775dfdd14
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64105a6d14215c0775dfdd14/-VX-cUYOLjHIg7QnWhRGG.jpeg
paper_authors.fullname: Jiwen Yu
paper_authors.type: user
paper_authors.user: VictorYuki
paper_authors._id: 678719ac5333dfbf8e206078
paper_authors.name: Yiran Qin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-21T11:49:26.310Z
paper_authors._id: 658a6c1399ed106ac8c822b1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/658a6c1399ed106ac8c822b1/Wk2KXCcK39rUvXx6mpmGD.jpeg
paper_authors.fullname: yiranqin
paper_authors.type: user
paper_authors.user: IranQin
paper_authors._id: 678719ac5333dfbf8e206079
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-21T10:08:34.083Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 678719ac5333dfbf8e20607a
paper_authors.name: Pengfei Wan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-17T14:42:53.121Z
paper_authors._id: 662f93942510ef5735d7ad00
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/662f93942510ef5735d7ad00/ZIDIPm63sncIHFTT5b0uR.png
paper_authors.fullname: magicwpf
paper_authors.type: user
paper_authors.user: magicwpf
paper_authors._id: 678719ac5333dfbf8e20607b
paper_authors.name: Di Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-21T10:08:59.963Z
paper_authors._id: 64bce15bafd1e46c5504ad38
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64bce15bafd1e46c5504ad38/vkEjiu-mIagKlrXzDH75o.png
paper_authors.fullname: Di Zhang
paper_authors.type: user
paper_authors.user: di-zhang-fdu
paper_authors._id: 678719ac5333dfbf8e20607c
paper_authors.name: Xihui Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-21T10:07:46.546Z
paper_authors._id: 65d5ec74cd05bc1eaa125040
paper_authors.avatarUrl: /avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
paper_authors.fullname: Xihui Liu
paper_authors.type: user
paper_authors.user: XihuiLiu
paper_title: GameFactory: Creating New Games with Generative Interactive Videos
paper_summary: Generative game engines have the potential to revolutionize game development
  by autonomously creating new content and reducing manual workload. However,
  existing video-based game generation methods fail to address the critical
  challenge of scene generalization, limiting their applicability to existing
  games with fixed styles and scenes. In this paper, we present GameFactory, a
  framework focused on exploring scene generalization in game video generation.
  To enable the creation of entirely new and diverse games, we leverage
  pre-trained video diffusion models trained on open-domain video data. To bridge
  the domain gap between open-domain priors and small-scale game dataset, we
  propose a multi-phase training strategy that decouples game style learning from
  action control, preserving open-domain generalization while achieving action
  controllability. Using Minecraft as our data source, we release GF-Minecraft, a
  high-quality and diversity action-annotated video dataset for research.
  Furthermore, we extend our framework to enable autoregressive
  action-controllable game video generation, allowing the production of
  unlimited-length interactive game videos. Experimental results demonstrate that
  GameFactory effectively generates open-domain, diverse, and action-controllable
  game videos, representing a significant step forward in AI-driven game
  generation. Our dataset and project page are publicly available at
  https://vvictoryuki.github.io/gamefactory/.
paper_ai_summary: GameFactory leverages pre-trained video diffusion models and a multi-phase training strategy to generate diverse, action-controllable game videos with scene generalization.
paper_ai_keywords: video diffusion models scene generalization action-controllability multi-phase training strategy action-annotated video dataset autoregressive model