title: AnimeGamer: Infinite Anime Life Simulation with Next Game State
    Prediction
summary: Recent advancements in image and video synthesis have opened up new promise
  in generative games. One particularly intriguing application is transforming
  characters from anime films into interactive, playable entities. This allows
  players to immerse themselves in the dynamic anime world as their favorite
  characters for life simulation through language instructions. Such games are
  defined as infinite game since they eliminate predetermined boundaries and
  fixed gameplay rules, where players can interact with the game world through
  open-ended language and experience ever-evolving storylines and environments.
  Recently, a pioneering approach for infinite anime life simulation employs
  large language models (LLMs) to translate multi-turn text dialogues into
  language instructions for image generation. However, it neglects historical
  visual context, leading to inconsistent gameplay. Furthermore, it only
  generates static images, failing to incorporate the dynamics necessary for an
  engaging gaming experience. In this work, we propose AnimeGamer, which is built
  upon Multimodal Large Language Models (MLLMs) to generate each game state,
  including dynamic animation shots that depict character movements and updates
  to character states, as illustrated in Figure 1. We introduce novel
  action-aware multimodal representations to represent animation shots, which can
  be decoded into high-quality video clips using a video diffusion model. By
  taking historical animation shot representations as context and predicting
  subsequent representations, AnimeGamer can generate games with contextual
  consistency and satisfactory dynamics. Extensive evaluations using both
  automated metrics and human evaluations demonstrate that AnimeGamer outperforms
  existing methods in various aspects of the gaming experience. Codes and
  checkpoints are available at https://github.com/TencentARC/AnimeGamer.
paper_authors._id: 67eca389e14049f5ff064ea6
paper_authors.name: Junhao Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:22:27.228Z
paper_authors._id: 6506b77a773ceaa8d52ecea1
paper_authors.avatarUrl: /avatars/0e769a0795063e1491c44760a4a83097.svg
paper_authors.fullname: CJH
paper_authors.type: user
paper_authors.user: Howe666
paper_authors._id: 67eca389e14049f5ff064ea7
paper_authors.name: Yuying Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-03T13:45:57.084Z
paper_authors._id: 6455cc8f654d8bccae50e4d4
paper_authors.avatarUrl: /avatars/506a9992e5bf52e06d37cc22e4b307c0.svg
paper_authors.fullname: Yuying Ge
paper_authors.type: user
paper_authors.user: tttoaster
paper_authors._id: 67eca389e14049f5ff064ea8
paper_authors.name: Yixiao Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-03T13:46:03.701Z
paper_authors._id: 640e9762b03f4cd29f58d982
paper_authors.avatarUrl: /avatars/81da37d628163fe3e094b247c7c3a3b5.svg
paper_authors.fullname: Yixiao Ge
paper_authors.type: user
paper_authors.user: yxgeee
paper_authors._id: 67eca389e14049f5ff064ea9
paper_authors.name: Jing Liao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-03T13:46:09.694Z
paper_authors._id: 65e77726767bfc7d109c45bf
paper_authors.avatarUrl: /avatars/24e68c86e06055ea1209598ba49ce8b9.svg
paper_authors.fullname: Jing Liao
paper_authors.type: user
paper_authors.user: CeciliaJL
paper_authors._id: 67eca389e14049f5ff064eaa
paper_authors.name: Ying Shan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-03T13:46:15.314Z
paper_authors._id: 63ca3ddc04c979828310bfcb
paper_authors.avatarUrl: /avatars/615e0d8622950b4408b40d550f02a894.svg
paper_authors.fullname: Ying Shan
paper_authors.type: user
paper_authors.user: yshan2u
paper_title: AnimeGamer: Infinite Anime Life Simulation with Next Game State
    Prediction
paper_summary: Recent advancements in image and video synthesis have opened up new promise
  in generative games. One particularly intriguing application is transforming
  characters from anime films into interactive, playable entities. This allows
  players to immerse themselves in the dynamic anime world as their favorite
  characters for life simulation through language instructions. Such games are
  defined as infinite game since they eliminate predetermined boundaries and
  fixed gameplay rules, where players can interact with the game world through
  open-ended language and experience ever-evolving storylines and environments.
  Recently, a pioneering approach for infinite anime life simulation employs
  large language models (LLMs) to translate multi-turn text dialogues into
  language instructions for image generation. However, it neglects historical
  visual context, leading to inconsistent gameplay. Furthermore, it only
  generates static images, failing to incorporate the dynamics necessary for an
  engaging gaming experience. In this work, we propose AnimeGamer, which is built
  upon Multimodal Large Language Models (MLLMs) to generate each game state,
  including dynamic animation shots that depict character movements and updates
  to character states, as illustrated in Figure 1. We introduce novel
  action-aware multimodal representations to represent animation shots, which can
  be decoded into high-quality video clips using a video diffusion model. By
  taking historical animation shot representations as context and predicting
  subsequent representations, AnimeGamer can generate games with contextual
  consistency and satisfactory dynamics. Extensive evaluations using both
  automated metrics and human evaluations demonstrate that AnimeGamer outperforms
  existing methods in various aspects of the gaming experience. Codes and
  checkpoints are available at https://github.com/TencentARC/AnimeGamer.
paper_ai_summary: AnimeGamer uses Multimodal Large Language Models (MLLMs) and a video diffusion model to generate contextually consistent, dynamic, and high-quality video clips for an infinite game simulation.
paper_ai_keywords: Multimodal Large Language Models MLLMs video diffusion model action-aware multimodal representations dynamic animation shots historical animation shot representations contextually consistent