CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot
  Interaction
Real-life robot navigation involves more than just reaching a destination; it
requires optimizing movements while addressing scenario-specific goals. An
intuitive way for humans to express these goals is through abstract cues like
verbal commands or rough sketches. Such human guidance may lack details or be
noisy. Nonetheless, we expect robots to navigate as intended. For robots to
interpret and execute these abstract instructions in line with human
expectations, they must share a common understanding of basic navigation
concepts with humans. To this end, we introduce CANVAS, a novel framework that
combines visual and linguistic instructions for commonsense-aware navigation.
Its success is driven by imitation learning, enabling the robot to learn from
human navigation behavior. We present COMMAND, a comprehensive dataset with
human-annotated navigation results, spanning over 48 hours and 219 km, designed
to train commonsense-aware navigation systems in simulated environments. Our
experiments show that CANVAS outperforms the strong rule-based system ROS
NavStack across all environments, demonstrating superior performance with noisy
instructions. Notably, in the orchard environment, where ROS NavStack records a
0% total success rate, CANVAS achieves a total success rate of 67%. CANVAS also
closely aligns with human demonstrations and commonsense constraints, even in
unseen environments. Furthermore, real-world deployment of CANVAS showcases
impressive Sim2Real transfer with a total success rate of 69%, highlighting the
potential of learning from human demonstrations in simulated environments for
real-world applications.
670391e647f71037735d4ab8
Suhwan Choi
admin_assigned
2024-10-07T09:19:23.886Z
632ced22ea6e62428aba1625
/avatars/0aae0ad956bdb2345732ca47fb28e3b6.svg
Suhwan Choi
user
MilkClouds
670391e647f71037735d4ab9
Yongjun Cho
claimed_verified
2024-10-07T09:16:57.884Z
65e972db5686ed1f5eff2327
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/er_WDGEq0HY8rpIg69v0G.png
Cho Yongjun
user
PurpleSand
670391e647f71037735d4aba
Minchan Kim
670391e647f71037735d4abb
Jaeyoon Jung
claimed_verified
2024-10-07T08:17:20.080Z
646484cfb90150b2706df03b
https://cdn-avatars.huggingface.co/v1/production/uploads/646484cfb90150b2706df03b/8ocSbXBSbrruhlhcxwzEt.png
Jaeyoon Jung
user
lastdefiance20
670391e647f71037735d4abc
Myunchul Joe
670391e647f71037735d4abd
Yubeen Park
admin_assigned
2024-10-07T09:20:02.994Z
646463124bf91229222a3217
/avatars/ca88658951b3aafa276e1cfa6403fc7c.svg
Yu Been Park
user
Unmanned-YuBeen
670391e647f71037735d4abe
Minseo Kim
claimed_verified
2024-10-13T20:18:08.733Z
64f22ee8a2fc808156dd1872
/avatars/45d9140acbfdbb0a8b044c2f9d3c4fad.svg
Kim Minseo
user
min99830
670391e647f71037735d4abf
Sungwoong Kim
claimed_verified
2024-10-09T07:38:18.819Z
662afff1ef7a4675bdf5bfb6
/avatars/511bc1c630dff30f7651ff8037110792.svg
sungwoong kim
user
sukim96
670391e647f71037735d4ac0
Sungjae Lee
670391e647f71037735d4ac1
Hwiseong Park
admin_assigned
2024-10-07T09:21:22.336Z
660f5b86f020db4b89aac9ef
/avatars/ffbb5711622420913686747d36901c50.svg
Hwiseong Park
user
wpiioos
670391e647f71037735d4ac2
Jiwan Chung
admin_assigned
2024-10-07T09:21:28.641Z
60d74d1affe0328e0167dc5f
/avatars/9b1a2df9402e9c26e1eb7c818af9bae0.svg
Jiwan Chung
user
jiwan-chung
670391e647f71037735d4ac3
Youngjae Yu
admin_assigned
2024-10-07T09:22:19.766Z
6504777fb1da3747a05160c4
/avatars/b777d98a5ff971ddb4c3e1060bb3e070.svg
Youngjae Yu
user
yjyu
CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot
  Interaction
Real-life robot navigation involves more than just reaching a destination; it
requires optimizing movements while addressing scenario-specific goals. An
intuitive way for humans to express these goals is through abstract cues like
verbal commands or rough sketches. Such human guidance may lack details or be
noisy. Nonetheless, we expect robots to navigate as intended. For robots to
interpret and execute these abstract instructions in line with human
expectations, they must share a common understanding of basic navigation
concepts with humans. To this end, we introduce CANVAS, a novel framework that
combines visual and linguistic instructions for commonsense-aware navigation.
Its success is driven by imitation learning, enabling the robot to learn from
human navigation behavior. We present COMMAND, a comprehensive dataset with
human-annotated navigation results, spanning over 48 hours and 219 km, designed
to train commonsense-aware navigation systems in simulated environments. Our
experiments show that CANVAS outperforms the strong rule-based system ROS
NavStack across all environments, demonstrating superior performance with noisy
instructions. Notably, in the orchard environment, where ROS NavStack records a
0% total success rate, CANVAS achieves a total success rate of 67%. CANVAS also
closely aligns with human demonstrations and commonsense constraints, even in
unseen environments. Furthermore, real-world deployment of CANVAS showcases
impressive Sim2Real transfer with a total success rate of 69%, highlighting the
potential of learning from human demonstrations in simulated environments for
real-world applications.
CANVAS, a framework combining visual and linguistic instructions, outperforms rule-based systems in robot navigation, especially with noisy inputs, through imitation learning and alignment with human demonstrations.
imitation learning CANVAS COMMAND commonsense-aware navigation ROS NavStack Sim2Real transfer