title: CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot
    Interaction
summary: Real-life robot navigation involves more than just reaching a destination; it
  requires optimizing movements while addressing scenario-specific goals. An
  intuitive way for humans to express these goals is through abstract cues like
  verbal commands or rough sketches. Such human guidance may lack details or be
  noisy. Nonetheless, we expect robots to navigate as intended. For robots to
  interpret and execute these abstract instructions in line with human
  expectations, they must share a common understanding of basic navigation
  concepts with humans. To this end, we introduce CANVAS, a novel framework that
  combines visual and linguistic instructions for commonsense-aware navigation.
  Its success is driven by imitation learning, enabling the robot to learn from
  human navigation behavior. We present COMMAND, a comprehensive dataset with
  human-annotated navigation results, spanning over 48 hours and 219 km, designed
  to train commonsense-aware navigation systems in simulated environments. Our
  experiments show that CANVAS outperforms the strong rule-based system ROS
  NavStack across all environments, demonstrating superior performance with noisy
  instructions. Notably, in the orchard environment, where ROS NavStack records a
  0% total success rate, CANVAS achieves a total success rate of 67%. CANVAS also
  closely aligns with human demonstrations and commonsense constraints, even in
  unseen environments. Furthermore, real-world deployment of CANVAS showcases
  impressive Sim2Real transfer with a total success rate of 69%, highlighting the
  potential of learning from human demonstrations in simulated environments for
  real-world applications.
paper_authors._id: 670391e647f71037735d4ab8
paper_authors.name: Suhwan Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-07T09:19:23.886Z
paper_authors._id: 632ced22ea6e62428aba1625
paper_authors.avatarUrl: /avatars/0aae0ad956bdb2345732ca47fb28e3b6.svg
paper_authors.fullname: Suhwan Choi
paper_authors.type: user
paper_authors.user: MilkClouds
paper_authors._id: 670391e647f71037735d4ab9
paper_authors.name: Yongjun Cho
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-07T09:16:57.884Z
paper_authors._id: 65e972db5686ed1f5eff2327
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/er_WDGEq0HY8rpIg69v0G.png
paper_authors.fullname: Cho Yongjun
paper_authors.type: user
paper_authors.user: PurpleSand
paper_authors._id: 670391e647f71037735d4aba
paper_authors.name: Minchan Kim
paper_authors._id: 670391e647f71037735d4abb
paper_authors.name: Jaeyoon Jung
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-07T08:17:20.080Z
paper_authors._id: 646484cfb90150b2706df03b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646484cfb90150b2706df03b/8ocSbXBSbrruhlhcxwzEt.png
paper_authors.fullname: Jaeyoon Jung
paper_authors.type: user
paper_authors.user: lastdefiance20
paper_authors._id: 670391e647f71037735d4abc
paper_authors.name: Myunchul Joe
paper_authors._id: 670391e647f71037735d4abd
paper_authors.name: Yubeen Park
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-07T09:20:02.994Z
paper_authors._id: 646463124bf91229222a3217
paper_authors.avatarUrl: /avatars/ca88658951b3aafa276e1cfa6403fc7c.svg
paper_authors.fullname: Yu Been Park
paper_authors.type: user
paper_authors.user: Unmanned-YuBeen
paper_authors._id: 670391e647f71037735d4abe
paper_authors.name: Minseo Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-13T20:18:08.733Z
paper_authors._id: 64f22ee8a2fc808156dd1872
paper_authors.avatarUrl: /avatars/45d9140acbfdbb0a8b044c2f9d3c4fad.svg
paper_authors.fullname: Kim Minseo
paper_authors.type: user
paper_authors.user: min99830
paper_authors._id: 670391e647f71037735d4abf
paper_authors.name: Sungwoong Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-09T07:38:18.819Z
paper_authors._id: 662afff1ef7a4675bdf5bfb6
paper_authors.avatarUrl: /avatars/511bc1c630dff30f7651ff8037110792.svg
paper_authors.fullname: sungwoong kim
paper_authors.type: user
paper_authors.user: sukim96
paper_authors._id: 670391e647f71037735d4ac0
paper_authors.name: Sungjae Lee
paper_authors._id: 670391e647f71037735d4ac1
paper_authors.name: Hwiseong Park
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-07T09:21:22.336Z
paper_authors._id: 660f5b86f020db4b89aac9ef
paper_authors.avatarUrl: /avatars/ffbb5711622420913686747d36901c50.svg
paper_authors.fullname: Hwiseong Park
paper_authors.type: user
paper_authors.user: wpiioos
paper_authors._id: 670391e647f71037735d4ac2
paper_authors.name: Jiwan Chung
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-07T09:21:28.641Z
paper_authors._id: 60d74d1affe0328e0167dc5f
paper_authors.avatarUrl: /avatars/9b1a2df9402e9c26e1eb7c818af9bae0.svg
paper_authors.fullname: Jiwan Chung
paper_authors.type: user
paper_authors.user: jiwan-chung
paper_authors._id: 670391e647f71037735d4ac3
paper_authors.name: Youngjae Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-07T09:22:19.766Z
paper_authors._id: 6504777fb1da3747a05160c4
paper_authors.avatarUrl: /avatars/b777d98a5ff971ddb4c3e1060bb3e070.svg
paper_authors.fullname: Youngjae Yu
paper_authors.type: user
paper_authors.user: yjyu
paper_title: CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot
    Interaction
paper_summary: Real-life robot navigation involves more than just reaching a destination; it
  requires optimizing movements while addressing scenario-specific goals. An
  intuitive way for humans to express these goals is through abstract cues like
  verbal commands or rough sketches. Such human guidance may lack details or be
  noisy. Nonetheless, we expect robots to navigate as intended. For robots to
  interpret and execute these abstract instructions in line with human
  expectations, they must share a common understanding of basic navigation
  concepts with humans. To this end, we introduce CANVAS, a novel framework that
  combines visual and linguistic instructions for commonsense-aware navigation.
  Its success is driven by imitation learning, enabling the robot to learn from
  human navigation behavior. We present COMMAND, a comprehensive dataset with
  human-annotated navigation results, spanning over 48 hours and 219 km, designed
  to train commonsense-aware navigation systems in simulated environments. Our
  experiments show that CANVAS outperforms the strong rule-based system ROS
  NavStack across all environments, demonstrating superior performance with noisy
  instructions. Notably, in the orchard environment, where ROS NavStack records a
  0% total success rate, CANVAS achieves a total success rate of 67%. CANVAS also
  closely aligns with human demonstrations and commonsense constraints, even in
  unseen environments. Furthermore, real-world deployment of CANVAS showcases
  impressive Sim2Real transfer with a total success rate of 69%, highlighting the
  potential of learning from human demonstrations in simulated environments for
  real-world applications.
paper_ai_keywords: imitation learning commonsense-aware navigation CANVAS COMMAND visual and linguistic instructions human-annotated navigation results Sim2Real transfer