title: Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
summary: We present Habitat 3.0: a simulation platform for studying collaborative
  human-robot tasks in home environments. Habitat 3.0 offers contributions across
  three dimensions: (1) Accurate humanoid simulation: addressing challenges in
  modeling complex deformable bodies and diversity in appearance and motion, all
  while ensuring high simulation speed. (2) Human-in-the-loop infrastructure:
  enabling real human interaction with simulated robots via mouse/keyboard or a
  VR interface, facilitating evaluation of robot policies with human input. (3)
  Collaborative tasks: studying two collaborative tasks, Social Navigation and
  Social Rearrangement. Social Navigation investigates a robot's ability to
  locate and follow humanoid avatars in unseen environments, whereas Social
  Rearrangement addresses collaboration between a humanoid and robot while
  rearranging a scene. These contributions allow us to study end-to-end learned
  and heuristic baselines for human-robot collaboration in-depth, as well as
  evaluate them with humans in the loop. Our experiments demonstrate that learned
  robot policies lead to efficient task completion when collaborating with unseen
  humanoid agents and human partners that might exhibit behaviors that the robot
  has not seen before. Additionally, we observe emergent behaviors during
  collaborative task execution, such as the robot yielding space when obstructing
  a humanoid agent, thereby allowing the effective completion of the task by the
  humanoid agent. Furthermore, our experiments using the human-in-the-loop tool
  demonstrate that our automated evaluation with humanoids can provide an
  indication of the relative ordering of different policies when evaluated with
  real human collaborators. Habitat 3.0 unlocks interesting new features in
  simulators for Embodied AI, and we hope it paves the way for a new frontier of
  embodied human-AI interaction capabilities.
paper_authors._id: 6537dcecd325b3f02fca4e4f
paper_authors.name: Xavier Puig
paper_authors._id: 6537dcecd325b3f02fca4e50
paper_authors.name: Eric Undersander
paper_authors._id: 6537dcecd325b3f02fca4e51
paper_authors.name: Andrew Szot
paper_authors._id: 6537dcecd325b3f02fca4e52
paper_authors.name: Mikael Dallaire Cote
paper_authors._id: 6537dcecd325b3f02fca4e53
paper_authors.name: Tsung-Yen Yang
paper_authors._id: 6537dcecd325b3f02fca4e54
paper_authors.name: Ruslan Partsey
paper_authors._id: 6537dcecd325b3f02fca4e55
paper_authors.name: Ruta Desai
paper_authors._id: 6537dcecd325b3f02fca4e56
paper_authors.name: Alexander William Clegg
paper_authors._id: 6537dcecd325b3f02fca4e57
paper_authors.name: Michal Hlavac
paper_authors._id: 6537dcecd325b3f02fca4e58
paper_authors.name: So Yeon Min
paper_authors._id: 6537dcecd325b3f02fca4e59
paper_authors.name: Vladimír Vondruš
paper_authors._id: 6537dcecd325b3f02fca4e5a
paper_authors.name: Theophile Gervet
paper_authors._id: 6537dcecd325b3f02fca4e5b
paper_authors.name: Vincent-Pierre Berges
paper_authors._id: 6537dcecd325b3f02fca4e5c
paper_authors.name: John M. Turner
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-12T16:34:36.738Z
paper_authors._id: 63ee68535f1300034dde7848
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676570521120-63ee68535f1300034dde7848.jpeg
paper_authors.fullname: John Turner
paper_authors.type: user
paper_authors.user: JohnTurner
paper_authors._id: 6537dcecd325b3f02fca4e5d
paper_authors.name: Oleksandr Maksymets
paper_authors._id: 6537dcecd325b3f02fca4e5e
paper_authors.name: Zsolt Kira
paper_authors._id: 6537dcecd325b3f02fca4e5f
paper_authors.name: Mrinal Kalakrishnan
paper_authors._id: 6537dcecd325b3f02fca4e60
paper_authors.name: Jitendra Malik
paper_authors._id: 6537dcecd325b3f02fca4e61
paper_authors.name: Devendra Singh Chaplot
paper_authors._id: 6537dcecd325b3f02fca4e62
paper_authors.name: Unnat Jain
paper_authors._id: 6537dcecd325b3f02fca4e63
paper_authors.name: Dhruv Batra
paper_authors._id: 6537dcecd325b3f02fca4e64
paper_authors.name: Akshara Rai
paper_authors._id: 6537dcecd325b3f02fca4e65
paper_authors.name: Roozbeh Mottaghi
paper_title: Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
paper_summary: We present Habitat 3.0: a simulation platform for studying collaborative
  human-robot tasks in home environments. Habitat 3.0 offers contributions across
  three dimensions: (1) Accurate humanoid simulation: addressing challenges in
  modeling complex deformable bodies and diversity in appearance and motion, all
  while ensuring high simulation speed. (2) Human-in-the-loop infrastructure:
  enabling real human interaction with simulated robots via mouse/keyboard or a
  VR interface, facilitating evaluation of robot policies with human input. (3)
  Collaborative tasks: studying two collaborative tasks, Social Navigation and
  Social Rearrangement. Social Navigation investigates a robot's ability to
  locate and follow humanoid avatars in unseen environments, whereas Social
  Rearrangement addresses collaboration between a humanoid and robot while
  rearranging a scene. These contributions allow us to study end-to-end learned
  and heuristic baselines for human-robot collaboration in-depth, as well as
  evaluate them with humans in the loop. Our experiments demonstrate that learned
  robot policies lead to efficient task completion when collaborating with unseen
  humanoid agents and human partners that might exhibit behaviors that the robot
  has not seen before. Additionally, we observe emergent behaviors during
  collaborative task execution, such as the robot yielding space when obstructing
  a humanoid agent, thereby allowing the effective completion of the task by the
  humanoid agent. Furthermore, our experiments using the human-in-the-loop tool
  demonstrate that our automated evaluation with humanoids can provide an
  indication of the relative ordering of different policies when evaluated with
  real human collaborators. Habitat 3.0 unlocks interesting new features in
  simulators for Embodied AI, and we hope it paves the way for a new frontier of
  embodied human-AI interaction capabilities.
paper_ai_summary: Habitat 3.0 provides a simulation platform for studying collaborative human-robot tasks with accurate humanoid simulation, human-in-the-loop infrastructure, and support for social navigation and rearrangement tasks.
paper_ai_keywords: humanoid simulation deformable bodies motion diversity simulation speed human-in-the-loop VR interface Social Navigation Social Rearrangement learned baselines heuristic baselines emergent behaviors