title: Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal
    Control
summary: We introduce Cosmos-Transfer, a conditional world generation model that can
  generate world simulations based on multiple spatial control inputs of various
  modalities such as segmentation, depth, and edge. In the design, the spatial
  conditional scheme is adaptive and customizable. It allows weighting different
  conditional inputs differently at different spatial locations. This enables
  highly controllable world generation and finds use in various world-to-world
  transfer use cases, including Sim2Real. We conduct extensive evaluations to
  analyze the proposed model and demonstrate its applications for Physical AI,
  including robotics Sim2Real and autonomous vehicle data enrichment. We further
  demonstrate an inference scaling strategy to achieve real-time world generation
  with an NVIDIA GB200 NVL72 rack. To help accelerate research development in the
  field, we open-source our models and code at
  https://github.com/nvidia-cosmos/cosmos-transfer1.
paper_authors._id: 67da2cbde5335651349e98c8
paper_authors.name: NVIDIA
paper_authors._id: 67da2cbde5335651349e98ca
paper_authors.name: Hassan Abu Alhaija
paper_authors._id: 67da2cbde5335651349e98cb
paper_authors.name: Jose Alvarez
paper_authors._id: 67da2cbde5335651349e98cc
paper_authors.name: Maciej Bala
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:33:45.678Z
paper_authors._id: 675304737c4876b7a1475695
paper_authors.avatarUrl: /avatars/7ee3c443f6a143b4a79d679fb7f60fe5.svg
paper_authors.fullname: Maciej Bala
paper_authors.type: user
paper_authors.user: mbalaNV
paper_authors._id: 67da2cbde5335651349e98cd
paper_authors.name: Tiffany Cai
paper_authors._id: 67da2cbde5335651349e98ce
paper_authors.name: Tianshi Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:33:56.311Z
paper_authors._id: 63e07dfaf0c75dfb876d9d71
paper_authors.avatarUrl: /avatars/8dab667735111e070cc6d44609dbdaf3.svg
paper_authors.fullname: Tianshi Cao
paper_authors.type: user
paper_authors.user: jeanlancel
paper_authors._id: 67da2cbde5335651349e98cf
paper_authors.name: Liz Cha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:34:03.422Z
paper_authors._id: 67a113431a84e2c54d77ca7c
paper_authors.avatarUrl: /avatars/b61e487b14eec1b29a4ccb61e13bb484.svg
paper_authors.fullname: Liz Cha
paper_authors.type: user
paper_authors.user: lizcha
paper_authors._id: 67da2cbde5335651349e98d0
paper_authors.name: Joshua Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:34:11.200Z
paper_authors._id: 640f46a506c3b5ca88404360
paper_authors.avatarUrl: /avatars/26c97b51d245bda62a0c3808aaba96a7.svg
paper_authors.fullname: Joshua Chen
paper_authors.type: user
paper_authors.user: JoshuaChen
paper_authors._id: 67da2cbde5335651349e98d1
paper_authors.name: Mike Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:34:18.080Z
paper_authors._id: 650874a5f36bb51c50f16294
paper_authors.avatarUrl: /avatars/112b7beb6e8c9d834349f99fefaca742.svg
paper_authors.fullname: Mike Chen
paper_authors.type: user
paper_authors.user: mikechen
paper_authors._id: 67da2cbde5335651349e98d2
paper_authors.name: Francesco Ferroni
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:34:24.297Z
paper_authors._id: 6353d3e95eac2d2efa7501f9
paper_authors.avatarUrl: /avatars/4b063f54000bed4bfb1bfcc3cde1a09e.svg
paper_authors.fullname: Francesco Ferroni
paper_authors.type: user
paper_authors.user: fferroni
paper_authors._id: 67da2cbde5335651349e98d3
paper_authors.name: Sanja Fidler
paper_authors._id: 67da2cbde5335651349e98d4
paper_authors.name: Dieter Fox
paper_authors._id: 67da2cbde5335651349e98d5
paper_authors.name: Yunhao Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:33:19.980Z
paper_authors._id: 64370de87cbe1e781fc5c7a2
paper_authors.avatarUrl: /avatars/98101739726ed797958e14f3d666933b.svg
paper_authors.fullname: Yunhao Ge
paper_authors.type: user
paper_authors.user: andyge
paper_authors._id: 67da2cbde5335651349e98d6
paper_authors.name: Jinwei Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:33:27.182Z
paper_authors._id: 647e8118770c299e56fc2bc8
paper_authors.avatarUrl: /avatars/adf80f3473dda42450148789ae5c208f.svg
paper_authors.fullname: Jinwei Gu
paper_authors.type: user
paper_authors.user: jwgu
paper_authors._id: 67da2cbde5335651349e98d7
paper_authors.name: Ali Hassani
paper_authors._id: 67da2cbde5335651349e98d8
paper_authors.name: Michael Isaev
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:34:43.605Z
paper_authors._id: 60d4c4616178079fbd2a0d1f
paper_authors.avatarUrl: /avatars/d3c2221fe1122fb04b383bc47c14e80d.svg
paper_authors.fullname: Michael Isaev
paper_authors.type: user
paper_authors.user: michael-isaev
paper_authors._id: 67da2cbde5335651349e98d9
paper_authors.name: Pooya Jannaty
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:34:49.164Z
paper_authors._id: 6771ca7396f52ab8156372c4
paper_authors.avatarUrl: /avatars/02adad495c9bd869e90db78d7baf41e0.svg
paper_authors.fullname: Pooya Jannaty
paper_authors.type: user
paper_authors.user: pjannaty
paper_authors._id: 67da2cbde5335651349e98da
paper_authors.name: Shiyi Lan
paper_authors._id: 67da2cbde5335651349e98db
paper_authors.name: Tobias Lasser
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:34:58.285Z
paper_authors._id: 66ec2efa1cd3794eb428806b
paper_authors.avatarUrl: /avatars/f00032293c1a2eb2f784f7881b7a744b.svg
paper_authors.fullname: Tobias Lasser
paper_authors.type: user
paper_authors.user: tlasser
paper_authors._id: 67da2cbde5335651349e98dc
paper_authors.name: Huan Ling
paper_authors._id: 67da2cbde5335651349e98dd
paper_authors.name: Ming-Yu Liu
paper_authors._id: 67da2cbde5335651349e98de
paper_authors.name: Xian Liu
paper_authors._id: 67da2cbde5335651349e98df
paper_authors.name: Yifan Lu
paper_authors._id: 67da2cbde5335651349e98e0
paper_authors.name: Alice Luo
paper_authors._id: 67da2cbde5335651349e98e1
paper_authors.name: Qianli Ma
paper_authors._id: 67da2cbde5335651349e98e2
paper_authors.name: Hanzi Mao
paper_authors._id: 67da2cbde5335651349e98e3
paper_authors.name: Fabio Ramos
paper_authors._id: 67da2cbde5335651349e98e4
paper_authors.name: Xuanchi Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T10:46:10.044Z
paper_authors._id: 658529d61c461dfe88afe8e8
paper_authors.avatarUrl: /avatars/a22c1b07d28c2662833c462c6537d835.svg
paper_authors.fullname: Xuanchi Ren
paper_authors.type: user
paper_authors.user: xrenaa
paper_authors._id: 67da2cbde5335651349e98e5
paper_authors.name: Tianchang Shen
paper_authors._id: 67da2cbde5335651349e98e6
paper_authors.name: Shitao Tang
paper_authors._id: 67da2cbde5335651349e98e7
paper_authors.name: Ting-Chun Wang
paper_authors._id: 67da2cbde5335651349e98e8
paper_authors.name: Jay Wu
paper_authors._id: 67da2cbde5335651349e98e9
paper_authors.name: Jiashu Xu
paper_authors._id: 67da2cbde5335651349e98ea
paper_authors.name: Stella Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:35:18.303Z
paper_authors._id: 645b420308d9a18f91e9748d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/5YOYvR2jgpKegopbMjVUF.jpeg
paper_authors.fullname: Stella Xu
paper_authors.type: user
paper_authors.user: sqx
paper_authors._id: 67da2cbde5335651349e98eb
paper_authors.name: Kevin Xie
paper_authors._id: 67da2cbde5335651349e98ec
paper_authors.name: Yuchong Ye
paper_authors._id: 67da2cbde5335651349e98ed
paper_authors.name: Xiaodong Yang
paper_authors._id: 67da2cbde5335651349e98ee
paper_authors.name: Xiaohui Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-19T13:35:10.244Z
paper_authors._id: 64d9cf178767727dff1da0c7
paper_authors.avatarUrl: /avatars/c2ec61b511754a8d312e2db3229215ec.svg
paper_authors.fullname: Xiaohui  ZENG
paper_authors.type: user
paper_authors.user: eccony
paper_authors._id: 67da2cbde5335651349e98ef
paper_authors.name: Yu Zeng
paper_title: Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal
    Control
paper_summary: We introduce Cosmos-Transfer, a conditional world generation model that can
  generate world simulations based on multiple spatial control inputs of various
  modalities such as segmentation, depth, and edge. In the design, the spatial
  conditional scheme is adaptive and customizable. It allows weighting different
  conditional inputs differently at different spatial locations. This enables
  highly controllable world generation and finds use in various world-to-world
  transfer use cases, including Sim2Real. We conduct extensive evaluations to
  analyze the proposed model and demonstrate its applications for Physical AI,
  including robotics Sim2Real and autonomous vehicle data enrichment. We further
  demonstrate an inference scaling strategy to achieve real-time world generation
  with an NVIDIA GB200 NVL72 rack. To help accelerate research development in the
  field, we open-source our models and code at
  https://github.com/nvidia-cosmos/cosmos-transfer1.
paper_ai_summary: Cosmos-Transfer is a conditional world generation model that uses adaptable spatial controls for segmentation, depth, and edge to generate real-time world simulations for Physical AI applications.
paper_ai_keywords: conditional world generation model spatial conditional scheme segmentation depth edge world-to-world transfer Sim2Real physical AI robotics Sim2Real autonomous vehicle data enrichment inference scaling strategy NVIDIA GB200 NVL72 rack