title: Vid2World: Crafting Video Diffusion Models to Interactive World Models
summary: World models, which predict transitions based on history observation and
  action sequences, have shown great promise in improving data efficiency for
  sequential decision making. However, existing world models often require
  extensive domain-specific training and still produce low-fidelity, coarse
  predictions, limiting their applicability in complex environments. In contrast,
  video diffusion models trained on large, internet-scale datasets have
  demonstrated impressive capabilities in generating high-quality videos that
  capture diverse real-world dynamics. In this work, we present Vid2World, a
  general approach for leveraging and transferring pre-trained video diffusion
  models into interactive world models. To bridge the gap, Vid2World performs
  casualization of a pre-trained video diffusion model by crafting its
  architecture and training objective to enable autoregressive generation.
  Furthermore, it introduces a causal action guidance mechanism to enhance action
  controllability in the resulting interactive world model. Extensive experiments
  in robot manipulation and game simulation domains show that our method offers a
  scalable and effective approach for repurposing highly capable video diffusion
  models to interactive world models.
paper_authors._id: 682d2b9ba006b93dbe79a333
paper_authors.name: Siqiao Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T18:04:10.245Z
paper_authors._id: 66b06fe4ce2224a4d066cc0a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66b06fe4ce2224a4d066cc0a/m7O-cdeRQVU1yc6WVmMzW.png
paper_authors.fullname: knightnemo
paper_authors.type: user
paper_authors.user: knightnemo
paper_authors._id: 682d2b9ba006b93dbe79a334
paper_authors.name: Jialong Wu
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-22T05:44:25.954Z
paper_authors._id: 643b866bff50448bcfc7d1d1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/q12v-BatVKimoi8q-coi-.jpeg
paper_authors.fullname: Jialong Wu
paper_authors.type: user
paper_authors.user: manchery
paper_authors._id: 682d2b9ba006b93dbe79a335
paper_authors.name: Qixing Zhou
paper_authors._id: 682d2b9ba006b93dbe79a336
paper_authors.name: Shangchen Miao
paper_authors._id: 682d2b9ba006b93dbe79a337
paper_authors.name: Mingsheng Long
paper_title: Vid2World: Crafting Video Diffusion Models to Interactive World Models
paper_summary: World models, which predict transitions based on history observation and
  action sequences, have shown great promise in improving data efficiency for
  sequential decision making. However, existing world models often require
  extensive domain-specific training and still produce low-fidelity, coarse
  predictions, limiting their applicability in complex environments. In contrast,
  video diffusion models trained on large, internet-scale datasets have
  demonstrated impressive capabilities in generating high-quality videos that
  capture diverse real-world dynamics. In this work, we present Vid2World, a
  general approach for leveraging and transferring pre-trained video diffusion
  models into interactive world models. To bridge the gap, Vid2World performs
  casualization of a pre-trained video diffusion model by crafting its
  architecture and training objective to enable autoregressive generation.
  Furthermore, it introduces a causal action guidance mechanism to enhance action
  controllability in the resulting interactive world model. Extensive experiments
  in robot manipulation and game simulation domains show that our method offers a
  scalable and effective approach for repurposing highly capable video diffusion
  models to interactive world models.
paper_ai_summary: Vid2World repurposes pre-trained video diffusion models into interactive world models via causalization and action guidance, enhancing action controllability and scalability in complex environments.
paper_ai_keywords: world models transitions history observation action sequences data efficiency sequential decision making low-fidelity coarse predictions video diffusion models internet-scale datasets high-quality videos real-world dynamics autoregressive generation causal action guidance robot manipulation game simulation