title: Genie: Generative Interactive Environments
summary: We introduce Genie, the first generative interactive environment trained in
  an unsupervised manner from unlabelled Internet videos. The model can be
  prompted to generate an endless variety of action-controllable virtual worlds
  described through text, synthetic images, photographs, and even sketches. At
  11B parameters, Genie can be considered a foundation world model. It is
  comprised of a spatiotemporal video tokenizer, an autoregressive dynamics
  model, and a simple and scalable latent action model. Genie enables users to
  act in the generated environments on a frame-by-frame basis despite training
  without any ground-truth action labels or other domain-specific requirements
  typically found in the world model literature. Further the resulting learned
  latent action space facilitates training agents to imitate behaviors from
  unseen videos, opening the path for training generalist agents of the future.
paper_authors._id: 65dc09e04b930db33ff49fdb
paper_authors.name: Jake Bruce
paper_authors._id: 65dc09e04b930db33ff49fdc
paper_authors.name: Michael Dennis
paper_authors._id: 65dc09e04b930db33ff49fdd
paper_authors.name: Ashley Edwards
paper_authors._id: 65dc09e04b930db33ff49fde
paper_authors.name: Jack Parker-Holder
paper_authors._id: 65dc09e04b930db33ff49fdf
paper_authors.name: Yuge Shi
paper_authors._id: 65dc09e04b930db33ff49fe0
paper_authors.name: Edward Hughes
paper_authors._id: 65dc09e04b930db33ff49fe1
paper_authors.name: Matthew Lai
paper_authors._id: 65dc09e04b930db33ff49fe2
paper_authors.name: Aditi Mavalankar
paper_authors._id: 65dc09e04b930db33ff49fe3
paper_authors.name: Richie Steigerwald
paper_authors._id: 65dc09e04b930db33ff49fe4
paper_authors.name: Chris Apps
paper_authors._id: 65dc09e04b930db33ff49fe5
paper_authors.name: Yusuf Aytar
paper_authors._id: 65dc09e04b930db33ff49fe6
paper_authors.name: Sarah Bechtle
paper_authors._id: 65dc09e04b930db33ff49fe7
paper_authors.name: Feryal Behbahani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-26T09:40:30.121Z
paper_authors._id: 64c197b7ec3c6181351b2c5d
paper_authors.avatarUrl: /avatars/f81fe98f68720b95885d4968b0e7cc0f.svg
paper_authors.fullname: Feryal Behbahani
paper_authors.type: user
paper_authors.user: ferya
paper_authors._id: 65dc09e04b930db33ff49fe8
paper_authors.name: Stephanie Chan
paper_authors._id: 65dc09e04b930db33ff49fe9
paper_authors.name: Nicolas Heess
paper_authors._id: 65dc09e04b930db33ff49fea
paper_authors.name: Lucy Gonzalez
paper_authors._id: 65dc09e04b930db33ff49feb
paper_authors.name: Simon Osindero
paper_authors._id: 65dc09e04b930db33ff49fec
paper_authors.name: Sherjil Ozair
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-26T09:41:10.671Z
paper_authors._id: 6311bda23a773344e068edc7
paper_authors.avatarUrl: /avatars/4a7cdbdac92ad6bcb91d2f7d52f03613.svg
paper_authors.fullname: Sherjil
paper_authors.type: user
paper_authors.user: sherjilozair
paper_authors._id: 65dc09e04b930db33ff49fed
paper_authors.name: Scott Reed
paper_authors._id: 65dc09e04b930db33ff49fee
paper_authors.name: Jingwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-26T09:47:44.724Z
paper_authors._id: 63ff47cc12c51862e5d9e2ce
paper_authors.avatarUrl: /avatars/fd08feaee58cebb3c4541eac8fe6bed6.svg
paper_authors.fullname: Jingwei Zhang
paper_authors.type: user
paper_authors.user: allanz
paper_authors._id: 65dc09e04b930db33ff49fef
paper_authors.name: Konrad Zolna
paper_authors._id: 65dc09e04b930db33ff49ff0
paper_authors.name: Jeff Clune
paper_authors._id: 65dc09e04b930db33ff49ff1
paper_authors.name: Nando de Freitas
paper_authors._id: 65dc09e04b930db33ff49ff2
paper_authors.name: Satinder Singh
paper_authors._id: 65dc09e04b930db33ff49ff3
paper_authors.name: Tim Rocktäschel
paper_title: Genie: Generative Interactive Environments
paper_summary: We introduce Genie, the first generative interactive environment trained in
  an unsupervised manner from unlabelled Internet videos. The model can be
  prompted to generate an endless variety of action-controllable virtual worlds
  described through text, synthetic images, photographs, and even sketches. At
  11B parameters, Genie can be considered a foundation world model. It is
  comprised of a spatiotemporal video tokenizer, an autoregressive dynamics
  model, and a simple and scalable latent action model. Genie enables users to
  act in the generated environments on a frame-by-frame basis despite training
  without any ground-truth action labels or other domain-specific requirements
  typically found in the world model literature. Further the resulting learned
  latent action space facilitates training agents to imitate behaviors from
  unseen videos, opening the path for training generalist agents of the future.
paper_ai_summary: Genie, a 11B parameter unsupervised generative model, creates action-controllable virtual worlds from unlabelled videos using spatiotemporal tokenization and autoregressive dynamics, enabling agent training from unseen video behaviors.
paper_ai_keywords: spatiotemporal video tokenizer autoregressive dynamics model latent action model world model foundation world model latent action space agent training