title: GenEx: Generating an Explorable World
summary: Understanding, navigating, and exploring the 3D physical real world has long
  been a central challenge in the development of artificial intelligence. In this
  work, we take a step toward this goal by introducing GenEx, a system capable of
  planning complex embodied world exploration, guided by its generative
  imagination that forms priors (expectations) about the surrounding
  environments. GenEx generates an entire 3D-consistent imaginative environment
  from as little as a single RGB image, bringing it to life through panoramic
  video streams. Leveraging scalable 3D world data curated from Unreal Engine,
  our generative model is rounded in the physical world. It captures a continuous
  360-degree environment with little effort, offering a boundless landscape for
  AI agents to explore and interact with. GenEx achieves high-quality world
  generation, robust loop consistency over long trajectories, and demonstrates
  strong 3D capabilities such as consistency and active 3D mapping. Powered by
  generative imagination of the world, GPT-assisted agents are equipped to
  perform complex embodied tasks, including both goal-agnostic exploration and
  goal-driven navigation. These agents utilize predictive expectation regarding
  unseen parts of the physical world to refine their beliefs, simulate different
  outcomes based on potential decisions, and make more informed choices. In
  summary, we demonstrate that GenEx provides a transformative platform for
  advancing embodied AI in imaginative spaces and brings potential for extending
  these capabilities to real-world exploration.
paper_authors._id: 675bcfacd0c3aeab39880d36
paper_authors.name: Taiming Lu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-13T08:56:00.579Z
paper_authors._id: 656a9b9f9496f21be8271f1b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/656a9b9f9496f21be8271f1b/e7p5kx2YNs_A1AJYSOlio.png
paper_authors.fullname: TaiMing
paper_authors.type: user
paper_authors.user: TaiMingLu
paper_authors._id: 675bcfacd0c3aeab39880d37
paper_authors.name: Tianmin Shu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T09:55:49.658Z
paper_authors._id: 618af79af281834c95311402
paper_authors.avatarUrl: /avatars/58ba086eb2309657d823b5768616eafe.svg
paper_authors.fullname: Tianmin Shu
paper_authors.type: user
paper_authors.user: tshu
paper_authors._id: 675bcfacd0c3aeab39880d38
paper_authors.name: Junfei Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-16T09:42:08.190Z
paper_authors._id: 64b5ba6060274cbb296d6288
paper_authors.avatarUrl: /avatars/67e0343954dda6e92ed3f6e7976f9f87.svg
paper_authors.fullname: Junfei Xiao
paper_authors.type: user
paper_authors.user: lambertxiao
paper_authors._id: 675bcfacd0c3aeab39880d39
paper_authors.name: Luoxin Ye
paper_authors._id: 675bcfacd0c3aeab39880d3a
paper_authors.name: Jiahao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-16T09:42:10.694Z
paper_authors._id: 64cbf523e3cc4a476d8291b6
paper_authors.avatarUrl: /avatars/825d7665db471e46921abad3319c2846.svg
paper_authors.fullname: Jiahao Wang
paper_authors.type: user
paper_authors.user: jiahaoplus
paper_authors._id: 675bcfacd0c3aeab39880d3b
paper_authors.name: Cheng Peng
paper_authors._id: 675bcfacd0c3aeab39880d3c
paper_authors.name: Chen Wei
paper_authors._id: 675bcfacd0c3aeab39880d3d
paper_authors.name: Daniel Khashabi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T09:55:06.806Z
paper_authors._id: 5f6540c65e78cc6b0ed3199d
paper_authors.avatarUrl: /avatars/0280d4df417855965a0964d22766c012.svg
paper_authors.fullname: Daniel Khashabi
paper_authors.type: user
paper_authors.user: danyaljj
paper_authors._id: 675bcfacd0c3aeab39880d3e
paper_authors.name: Rama Chellappa
paper_authors._id: 675bcfacd0c3aeab39880d3f
paper_authors.name: Alan Yuille
paper_authors._id: 675bcfacd0c3aeab39880d40
paper_authors.name: Jieneng Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T09:54:46.355Z
paper_authors._id: 660c9ac4b202fcf3892f62fa
paper_authors.avatarUrl: /avatars/7314fd5f3f642096d0e37d3194f1aa7e.svg
paper_authors.fullname: Jieneng Chen
paper_authors.type: user
paper_authors.user: jienengchen
paper_title: GenEx: Generating an Explorable World
paper_summary: Understanding, navigating, and exploring the 3D physical real world has long
  been a central challenge in the development of artificial intelligence. In this
  work, we take a step toward this goal by introducing GenEx, a system capable of
  planning complex embodied world exploration, guided by its generative
  imagination that forms priors (expectations) about the surrounding
  environments. GenEx generates an entire 3D-consistent imaginative environment
  from as little as a single RGB image, bringing it to life through panoramic
  video streams. Leveraging scalable 3D world data curated from Unreal Engine,
  our generative model is rounded in the physical world. It captures a continuous
  360-degree environment with little effort, offering a boundless landscape for
  AI agents to explore and interact with. GenEx achieves high-quality world
  generation, robust loop consistency over long trajectories, and demonstrates
  strong 3D capabilities such as consistency and active 3D mapping. Powered by
  generative imagination of the world, GPT-assisted agents are equipped to
  perform complex embodied tasks, including both goal-agnostic exploration and
  goal-driven navigation. These agents utilize predictive expectation regarding
  unseen parts of the physical world to refine their beliefs, simulate different
  outcomes based on potential decisions, and make more informed choices. In
  summary, we demonstrate that GenEx provides a transformative platform for
  advancing embodied AI in imaginative spaces and brings potential for extending
  these capabilities to real-world exploration.
paper_ai_summary: GenEx generates 3D environments from a single image, enabling AI agents to explore and interact with a consistent, expansive space through guided generative imagination.
paper_ai_keywords: Generative imagination panoramic video streams Unreal Engine world generation loop consistency 3D mapping goal-agnostic exploration goal-driven navigation predictive expectation informed choices