Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal
  Control
We introduce Cosmos-Transfer, a conditional world generation model that can
generate world simulations based on multiple spatial control inputs of various
modalities such as segmentation, depth, and edge. In the design, the spatial
conditional scheme is adaptive and customizable. It allows weighting different
conditional inputs differently at different spatial locations. This enables
highly controllable world generation and finds use in various world-to-world
transfer use cases, including Sim2Real. We conduct extensive evaluations to
analyze the proposed model and demonstrate its applications for Physical AI,
including robotics Sim2Real and autonomous vehicle data enrichment. We further
demonstrate an inference scaling strategy to achieve real-time world generation
with an NVIDIA GB200 NVL72 rack. To help accelerate research development in the
field, we open-source our models and code at
https://github.com/nvidia-cosmos/cosmos-transfer1.
67da2cbde5335651349e98c8
NVIDIA
67da2cbde5335651349e98ca
Hassan Abu Alhaija
67da2cbde5335651349e98cb
Jose Alvarez
67da2cbde5335651349e98cc
Maciej Bala
admin_assigned
2025-03-19T13:33:45.678Z
675304737c4876b7a1475695
/avatars/7ee3c443f6a143b4a79d679fb7f60fe5.svg
Maciej Bala
user
mbalaNV
67da2cbde5335651349e98cd
Tiffany Cai
67da2cbde5335651349e98ce
Tianshi Cao
admin_assigned
2025-03-19T13:33:56.311Z
63e07dfaf0c75dfb876d9d71
/avatars/8dab667735111e070cc6d44609dbdaf3.svg
Tianshi Cao
user
jeanlancel
67da2cbde5335651349e98cf
Liz Cha
admin_assigned
2025-03-19T13:34:03.422Z
67a113431a84e2c54d77ca7c
/avatars/b61e487b14eec1b29a4ccb61e13bb484.svg
Liz Cha
user
lizcha
67da2cbde5335651349e98d0
Joshua Chen
admin_assigned
2025-03-19T13:34:11.200Z
640f46a506c3b5ca88404360
/avatars/26c97b51d245bda62a0c3808aaba96a7.svg
Joshua Chen
user
JoshuaChen
67da2cbde5335651349e98d1
Mike Chen
admin_assigned
2025-03-19T13:34:18.080Z
650874a5f36bb51c50f16294
/avatars/112b7beb6e8c9d834349f99fefaca742.svg
Mike Chen
user
mikechen
67da2cbde5335651349e98d2
Francesco Ferroni
admin_assigned
2025-03-19T13:34:24.297Z
6353d3e95eac2d2efa7501f9
/avatars/4b063f54000bed4bfb1bfcc3cde1a09e.svg
Francesco Ferroni
user
fferroni
67da2cbde5335651349e98d3
Sanja Fidler
67da2cbde5335651349e98d4
Dieter Fox
67da2cbde5335651349e98d5
Yunhao Ge
admin_assigned
2025-03-19T13:33:19.980Z
64370de87cbe1e781fc5c7a2
/avatars/98101739726ed797958e14f3d666933b.svg
Yunhao Ge
user
andyge
67da2cbde5335651349e98d6
Jinwei Gu
admin_assigned
2025-03-19T13:33:27.182Z
647e8118770c299e56fc2bc8
/avatars/adf80f3473dda42450148789ae5c208f.svg
Jinwei Gu
user
jwgu
67da2cbde5335651349e98d7
Ali Hassani
67da2cbde5335651349e98d8
Michael Isaev
admin_assigned
2025-03-19T13:34:43.605Z
60d4c4616178079fbd2a0d1f
/avatars/d3c2221fe1122fb04b383bc47c14e80d.svg
Michael Isaev
user
michael-isaev
67da2cbde5335651349e98d9
Pooya Jannaty
admin_assigned
2025-03-19T13:34:49.164Z
6771ca7396f52ab8156372c4
/avatars/02adad495c9bd869e90db78d7baf41e0.svg
Pooya Jannaty
user
pjannaty
67da2cbde5335651349e98da
Shiyi Lan
67da2cbde5335651349e98db
Tobias Lasser
admin_assigned
2025-03-19T13:34:58.285Z
66ec2efa1cd3794eb428806b
/avatars/f00032293c1a2eb2f784f7881b7a744b.svg
Tobias Lasser
user
tlasser
67da2cbde5335651349e98dc
Huan Ling
67da2cbde5335651349e98dd
Ming-Yu Liu
67da2cbde5335651349e98de
Xian Liu
67da2cbde5335651349e98df
Yifan Lu
67da2cbde5335651349e98e0
Alice Luo
67da2cbde5335651349e98e1
Qianli Ma
67da2cbde5335651349e98e2
Hanzi Mao
67da2cbde5335651349e98e3
Fabio Ramos
67da2cbde5335651349e98e4
Xuanchi Ren
claimed_verified
2025-03-20T10:46:10.044Z
658529d61c461dfe88afe8e8
/avatars/a22c1b07d28c2662833c462c6537d835.svg
Xuanchi Ren
user
xrenaa
67da2cbde5335651349e98e5
Tianchang Shen
67da2cbde5335651349e98e6
Shitao Tang
67da2cbde5335651349e98e7
Ting-Chun Wang
67da2cbde5335651349e98e8
Jay Wu
67da2cbde5335651349e98e9
Jiashu Xu
67da2cbde5335651349e98ea
Stella Xu
admin_assigned
2025-03-19T13:35:18.303Z
645b420308d9a18f91e9748d
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/5YOYvR2jgpKegopbMjVUF.jpeg
Stella Xu
user
sqx
67da2cbde5335651349e98eb
Kevin Xie
67da2cbde5335651349e98ec
Yuchong Ye
67da2cbde5335651349e98ed
Xiaodong Yang
67da2cbde5335651349e98ee
Xiaohui Zeng
admin_assigned
2025-03-19T13:35:10.244Z
64d9cf178767727dff1da0c7
/avatars/c2ec61b511754a8d312e2db3229215ec.svg
Xiaohui  ZENG
user
eccony
67da2cbde5335651349e98ef
Yu Zeng
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal
  Control
We introduce Cosmos-Transfer, a conditional world generation model that can
generate world simulations based on multiple spatial control inputs of various
modalities such as segmentation, depth, and edge. In the design, the spatial
conditional scheme is adaptive and customizable. It allows weighting different
conditional inputs differently at different spatial locations. This enables
highly controllable world generation and finds use in various world-to-world
transfer use cases, including Sim2Real. We conduct extensive evaluations to
analyze the proposed model and demonstrate its applications for Physical AI,
including robotics Sim2Real and autonomous vehicle data enrichment. We further
demonstrate an inference scaling strategy to achieve real-time world generation
with an NVIDIA GB200 NVL72 rack. To help accelerate research development in the
field, we open-source our models and code at
https://github.com/nvidia-cosmos/cosmos-transfer1.
Cosmos-Transfer is a conditional world generation model that uses adaptable spatial controls for segmentation, depth, and edge to generate real-time world simulations for Physical AI applications.
conditional world generation model spatial conditional scheme segmentation depth edge world-to-world transfer Sim2Real physical AI robotics Sim2Real autonomous vehicle data enrichment inference scaling strategy NVIDIA GB200 NVL72 rack