HomeRobot: Open-Vocabulary Mobile Manipulation
HomeRobot (noun): An affordable compliant robot that navigates homes and
manipulates a wide range of objects in order to complete everyday tasks.
Open-Vocabulary Mobile Manipulation (OVMM) is the problem of picking any object
in any unseen environment, and placing it in a commanded location. This is a
foundational challenge for robots to be useful assistants in human
environments, because it involves tackling sub-problems from across robotics:
perception, language understanding, navigation, and manipulation are all
essential to OVMM. In addition, integration of the solutions to these
sub-problems poses its own substantial challenges. To drive research in this
area, we introduce the HomeRobot OVMM benchmark, where an agent navigates
household environments to grasp novel objects and place them on target
receptacles. HomeRobot has two components: a simulation component, which uses a
large and diverse curated object set in new, high-quality multi-room home
environments; and a real-world component, providing a software stack for the
low-cost Hello Robot Stretch to encourage replication of real-world experiments
across labs. We implement both reinforcement learning and heuristic
(model-based) baselines and show evidence of sim-to-real transfer. Our
baselines achieve a 20% success rate in the real world; our experiments
identify ways future research work improve performance. See videos on our
website: https://ovmm.github.io/.
64927a832aa84c46de6ba8ea
Sriram Yenamandra
claimed_verified
2023-06-21T09:16:41.054Z
640d3ece4b5450a208b275fd
/avatars/669e161448c0895b856065646d68c24c.svg
sriram
user
yvsriram
64927a832aa84c46de6ba8eb
Arun Ramachandran
admin_assigned
2023-06-21T15:33:11.751Z
63ed2501f765928ceead06cb
https://cdn-avatars.huggingface.co/v1/production/uploads/63ed2501f765928ceead06cb/UffowP9cBW5CZoLo0BhP5.jpeg
Arun Ramachandran
user
JHurricane96
64927a832aa84c46de6ba8ec
Karmesh Yadav
admin_assigned
2023-06-21T15:33:02.356Z
63ed4eac679c2cc40ab86838
/avatars/f9a0d4af63dcc713e77c3533d1809699.svg
Karmesh Yadav
user
ykarmesh
64927a832aa84c46de6ba8ed
Austin Wang
64927a832aa84c46de6ba8ee
Mukul Khanna
admin_assigned
2023-06-21T15:34:00.351Z
6387a0328b3c55fea3e4a13d
https://cdn-avatars.huggingface.co/v1/production/uploads/6387a0328b3c55fea3e4a13d/uxMoiqxD0GLKV95ItECJr.jpeg
Mukul Khanna
user
mukulkhanna
64927a832aa84c46de6ba8ef
Theophile Gervet
admin_assigned
2023-06-21T15:34:20.651Z
644e822cd6001776ed7700e4
/avatars/58a72e6bf8ede3cee3de54873231fd3e.svg
Theophile Gervet
user
TheophileGervet
64927a832aa84c46de6ba8f0
Tsung-Yen Yang
64927a832aa84c46de6ba8f1
Vidhi Jain
admin_assigned
2023-06-21T15:35:03.268Z
6335dd2717cccb7421bb0def
https://cdn-avatars.huggingface.co/v1/production/uploads/6335dd2717cccb7421bb0def/yFjtuIjyyh6MY42pKLLjw.png
Vidhi Jain
user
vidhij
64927a832aa84c46de6ba8f2
Alexander William Clegg
64927a832aa84c46de6ba8f3
John Turner
admin_assigned
2023-06-21T15:35:59.683Z
63ee68535f1300034dde7848
https://cdn-avatars.huggingface.co/v1/production/uploads/1676570521120-63ee68535f1300034dde7848.jpeg
John Turner
user
JohnTurner
64927a832aa84c46de6ba8f4
Zsolt Kira
admin_assigned
2023-06-21T15:36:16.491Z
64716e6e085607e2f5842446
/avatars/f69a0869eb6adfdbfd746a56390de2df.svg
Zsolt Kira
user
zk-danti
64927a832aa84c46de6ba8f5
Manolis Savva
admin_assigned
2023-06-21T15:37:04.370Z
639f45949f1f2baab2f34c79
/avatars/6035ba3b8ccecda262316076dfd1ff60.svg
Manolis Savva
user
msavva
64927a832aa84c46de6ba8f6
Angel Chang
admin_assigned
2023-06-21T15:37:25.335Z
6363d5d92e11f41bc819176e
/avatars/516bea7f7da973bfcc992df9a63727ca.svg
Angel Chang
user
angelxuanchang
64927a832aa84c46de6ba8f7
Devendra Singh Chaplot
64927a832aa84c46de6ba8f8
Dhruv Batra
admin_assigned
2023-06-21T15:38:02.559Z
63ed1e57f765928ceeacbe02
/avatars/e5a7428bb0acb43a240f1c4a8c1a5639.svg
Dhruv Batra
user
dbatra
64927a832aa84c46de6ba8f9
Roozbeh Mottaghi
64927a832aa84c46de6ba8fa
Yonatan Bisk
admin_assigned
2023-06-21T15:38:36.509Z
63180f3c879073cf83f135b0
https://cdn-avatars.huggingface.co/v1/production/uploads/63180f3c879073cf83f135b0/0ycXVKOywlU1hcvQ4MDws.jpeg
Yonatan Bisk
user
ybisk
64927a832aa84c46de6ba8fb
Chris Paxton
claimed_verified
2023-06-21T20:46:30.825Z
62b13b28b7053ab38d2d5f96
https://cdn-avatars.huggingface.co/v1/production/uploads/62b13b28b7053ab38d2d5f96/V6780Omima7sNW5E4TEeW.png
Chris Paxton
user
cpaxton
HomeRobot: Open-Vocabulary Mobile Manipulation
HomeRobot (noun): An affordable compliant robot that navigates homes and
manipulates a wide range of objects in order to complete everyday tasks.
Open-Vocabulary Mobile Manipulation (OVMM) is the problem of picking any object
in any unseen environment, and placing it in a commanded location. This is a
foundational challenge for robots to be useful assistants in human
environments, because it involves tackling sub-problems from across robotics:
perception, language understanding, navigation, and manipulation are all
essential to OVMM. In addition, integration of the solutions to these
sub-problems poses its own substantial challenges. To drive research in this
area, we introduce the HomeRobot OVMM benchmark, where an agent navigates
household environments to grasp novel objects and place them on target
receptacles. HomeRobot has two components: a simulation component, which uses a
large and diverse curated object set in new, high-quality multi-room home
environments; and a real-world component, providing a software stack for the
low-cost Hello Robot Stretch to encourage replication of real-world experiments
across labs. We implement both reinforcement learning and heuristic
(model-based) baselines and show evidence of sim-to-real transfer. Our
baselines achieve a 20% success rate in the real world; our experiments
identify ways future research work improve performance. See videos on our
website: https://ovmm.github.io/.
The HomeRobot OVMM benchmark evaluates robots' ability to manipulate unseen objects in new environments using a combination of simulated and real-world testing.
Open-Vocabulary Mobile Manipulation percpetion language understanding navigation manipulation reinforcement learning heuristic (model-based) sim-to-real transfer HomeRobot Hello Robot Stretch