title: Generative World Explorer
summary: Planning with partial observation is a central challenge in embodied AI. A
  majority of prior works have tackled this challenge by developing agents that
  physically explore their environment to update their beliefs about the world
  state.In contrast, humans can imagine unseen parts of the world
  through a mental exploration and revise their beliefs with imagined
  observations. Such updated beliefs can allow them to make more informed
  decisions, without necessitating the physical exploration of the world at all
  times. To achieve this human-like ability, we introduce the Generative
  World Explorer (Genex), an egocentric world exploration framework that allows
  an agent to mentally explore a large-scale 3D world (e.g., urban scenes) and
  acquire imagined observations to update its belief. This updated belief will
  then help the agent to make a more informed decision at the current step. To
  train Genex, we create a synthetic urban scene dataset, Genex-DB.
  Our experimental results demonstrate that (1) Genex can generate
  high-quality and consistent observations during long-horizon exploration of a
  large virtual physical world and (2) the beliefs updated with the generated
  observations can inform an existing decision-making model (e.g., an LLM agent)
  to make better plans.
paper_authors._id: 673bfe4c2bf4163ef13f497f
paper_authors.name: Taiming Lu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-19T08:39:37.063Z
paper_authors._id: 656a9b9f9496f21be8271f1b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/656a9b9f9496f21be8271f1b/e7p5kx2YNs_A1AJYSOlio.png
paper_authors.fullname: TaiMing
paper_authors.type: user
paper_authors.user: TaiMingLu
paper_authors._id: 673bfe4c2bf4163ef13f4980
paper_authors.name: Tianmin Shu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T13:07:10.536Z
paper_authors._id: 618af79af281834c95311402
paper_authors.avatarUrl: /avatars/58ba086eb2309657d823b5768616eafe.svg
paper_authors.fullname: Tianmin Shu
paper_authors.type: user
paper_authors.user: tshu
paper_authors._id: 673bfe4c2bf4163ef13f4981
paper_authors.name: Alan Yuille
paper_authors._id: 673bfe4c2bf4163ef13f4982
paper_authors.name: Daniel Khashabi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T13:07:23.603Z
paper_authors._id: 5f6540c65e78cc6b0ed3199d
paper_authors.avatarUrl: /avatars/0280d4df417855965a0964d22766c012.svg
paper_authors.fullname: Daniel Khashabi
paper_authors.type: user
paper_authors.user: danyaljj
paper_authors._id: 673bfe4c2bf4163ef13f4983
paper_authors.name: Jieneng Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-19T08:39:33.842Z
paper_authors._id: 660c9ac4b202fcf3892f62fa
paper_authors.avatarUrl: /avatars/7314fd5f3f642096d0e37d3194f1aa7e.svg
paper_authors.fullname: Jieneng Chen
paper_authors.type: user
paper_authors.user: jienengchen
paper_title: Generative World Explorer
paper_summary: Planning with partial observation is a central challenge in embodied AI. A
  majority of prior works have tackled this challenge by developing agents that
  physically explore their environment to update their beliefs about the world
  state.In contrast, humans can imagine unseen parts of the world
  through a mental exploration and revise their beliefs with imagined
  observations. Such updated beliefs can allow them to make more informed
  decisions, without necessitating the physical exploration of the world at all
  times. To achieve this human-like ability, we introduce the Generative
  World Explorer (Genex), an egocentric world exploration framework that allows
  an agent to mentally explore a large-scale 3D world (e.g., urban scenes) and
  acquire imagined observations to update its belief. This updated belief will
  then help the agent to make a more informed decision at the current step. To
  train Genex, we create a synthetic urban scene dataset, Genex-DB.
  Our experimental results demonstrate that (1) Genex can generate
  high-quality and consistent observations during long-horizon exploration of a
  large virtual physical world and (2) the beliefs updated with the generated
  observations can inform an existing decision-making model (e.g., an LLM agent)
  to make better plans.
paper_ai_summary: Generative World Explorer (Genex) enables agents to mentally explore 3D environments using imagined observations, updating their beliefs without physical exploration to improve decision-making.
paper_ai_keywords: Generative World Explorer Genex egocentric world exploration belief updating imagined observations large-scale 3D world synthesis Genex-DB long-horizon exploration decision-making model LLM agent