Genie: Generative Interactive Environments
We introduce Genie, the first generative interactive environment trained in
an unsupervised manner from unlabelled Internet videos. The model can be
prompted to generate an endless variety of action-controllable virtual worlds
described through text, synthetic images, photographs, and even sketches. At
11B parameters, Genie can be considered a foundation world model. It is
comprised of a spatiotemporal video tokenizer, an autoregressive dynamics
model, and a simple and scalable latent action model. Genie enables users to
act in the generated environments on a frame-by-frame basis despite training
without any ground-truth action labels or other domain-specific requirements
typically found in the world model literature. Further the resulting learned
latent action space facilitates training agents to imitate behaviors from
unseen videos, opening the path for training generalist agents of the future.
65dc09e04b930db33ff49fdb
Jake Bruce
65dc09e04b930db33ff49fdc
Michael Dennis
65dc09e04b930db33ff49fdd
Ashley Edwards
65dc09e04b930db33ff49fde
Jack Parker-Holder
65dc09e04b930db33ff49fdf
Yuge Shi
65dc09e04b930db33ff49fe0
Edward Hughes
65dc09e04b930db33ff49fe1
Matthew Lai
65dc09e04b930db33ff49fe2
Aditi Mavalankar
65dc09e04b930db33ff49fe3
Richie Steigerwald
65dc09e04b930db33ff49fe4
Chris Apps
65dc09e04b930db33ff49fe5
Yusuf Aytar
65dc09e04b930db33ff49fe6
Sarah Bechtle
65dc09e04b930db33ff49fe7
Feryal Behbahani
admin_assigned
2024-02-26T09:40:30.121Z
64c197b7ec3c6181351b2c5d
/avatars/f81fe98f68720b95885d4968b0e7cc0f.svg
Feryal Behbahani
user
ferya
65dc09e04b930db33ff49fe8
Stephanie Chan
65dc09e04b930db33ff49fe9
Nicolas Heess
65dc09e04b930db33ff49fea
Lucy Gonzalez
65dc09e04b930db33ff49feb
Simon Osindero
65dc09e04b930db33ff49fec
Sherjil Ozair
admin_assigned
2024-02-26T09:41:10.671Z
6311bda23a773344e068edc7
/avatars/4a7cdbdac92ad6bcb91d2f7d52f03613.svg
Sherjil
user
sherjilozair
65dc09e04b930db33ff49fed
Scott Reed
65dc09e04b930db33ff49fee
Jingwei Zhang
admin_assigned
2024-02-26T09:47:44.724Z
63ff47cc12c51862e5d9e2ce
/avatars/fd08feaee58cebb3c4541eac8fe6bed6.svg
Jingwei Zhang
user
allanz
65dc09e04b930db33ff49fef
Konrad Zolna
65dc09e04b930db33ff49ff0
Jeff Clune
65dc09e04b930db33ff49ff1
Nando de Freitas
65dc09e04b930db33ff49ff2
Satinder Singh
65dc09e04b930db33ff49ff3
Tim Rocktäschel
Genie: Generative Interactive Environments
We introduce Genie, the first generative interactive environment trained in
an unsupervised manner from unlabelled Internet videos. The model can be
prompted to generate an endless variety of action-controllable virtual worlds
described through text, synthetic images, photographs, and even sketches. At
11B parameters, Genie can be considered a foundation world model. It is
comprised of a spatiotemporal video tokenizer, an autoregressive dynamics
model, and a simple and scalable latent action model. Genie enables users to
act in the generated environments on a frame-by-frame basis despite training
without any ground-truth action labels or other domain-specific requirements
typically found in the world model literature. Further the resulting learned
latent action space facilitates training agents to imitate behaviors from
unseen videos, opening the path for training generalist agents of the future.
Genie, a 11B parameter unsupervised generative model, creates action-controllable virtual worlds from unlabelled videos using spatiotemporal tokenization and autoregressive dynamics, enabling agent training from unseen video behaviors.
spatiotemporal video tokenizer autoregressive dynamics model latent action model world model foundation world model latent action space agent training