title: UniGoal: Towards Universal Zero-shot Goal-oriented Navigation
summary: In this paper, we propose a general framework for universal zero-shot
  goal-oriented navigation. Existing zero-shot methods build inference framework
  upon large language models (LLM) for specific tasks, which differs a lot in
  overall pipeline and fails to generalize across different types of goal.
  Towards the aim of universal zero-shot navigation, we propose a uniform graph
  representation to unify different goals, including object category, instance
  image and text description. We also convert the observation of agent into an
  online maintained scene graph. With this consistent scene and goal
  representation, we preserve most structural information compared with pure text
  and are able to leverage LLM for explicit graph-based reasoning. Specifically,
  we conduct graph matching between the scene graph and goal graph at each time
  instant and propose different strategies to generate long-term goal of
  exploration according to different matching states. The agent first iteratively
  searches subgraph of goal when zero-matched. With partial matching, the agent
  then utilizes coordinate projection and anchor pair alignment to infer the goal
  location. Finally scene graph correction and goal verification are applied for
  perfect matching. We also present a blacklist mechanism to enable robust switch
  between stages. Extensive experiments on several benchmarks show that our
  UniGoal achieves state-of-the-art zero-shot performance on three studied
  navigation tasks with a single model, even outperforming task-specific
  zero-shot methods and supervised universal methods.
paper_authors._id: 67d39f4828221b583a33be2c
paper_authors.name: Hang Yin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:05:49.754Z
paper_authors._id: 6651034c05fb627b10f8abe5
paper_authors.avatarUrl: /avatars/4fc0835d5956744295e515f8ff83b3df.svg
paper_authors.fullname: Hang Yin
paper_authors.type: user
paper_authors.user: hangyin
paper_authors._id: 67d39f4828221b583a33be2d
paper_authors.name: Xiuwei Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:05:43.448Z
paper_authors._id: 648ac65fd044b25978015634
paper_authors.avatarUrl: /avatars/2278a66fdc953220e9f8fc0ccce3ff00.svg
paper_authors.fullname: Xiuwei Xu
paper_authors.type: user
paper_authors.user: xuxw98
paper_authors._id: 67d39f4828221b583a33be2e
paper_authors.name: Lingqing Zhao
paper_authors._id: 67d39f4828221b583a33be2f
paper_authors.name: Ziwei Wang
paper_authors._id: 67d39f4828221b583a33be30
paper_authors.name: Jie Zhou
paper_authors._id: 67d39f4828221b583a33be31
paper_authors.name: Jiwen Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:04:42.455Z
paper_authors._id: 66c44203ea476bea05e9fcd7
paper_authors.avatarUrl: /avatars/b061eebec609446e669f5ad6365959f9.svg
paper_authors.fullname: lu
paper_authors.type: user
paper_authors.user: jiwenlu
paper_title: UniGoal: Towards Universal Zero-shot Goal-oriented Navigation
paper_summary: In this paper, we propose a general framework for universal zero-shot
  goal-oriented navigation. Existing zero-shot methods build inference framework
  upon large language models (LLM) for specific tasks, which differs a lot in
  overall pipeline and fails to generalize across different types of goal.
  Towards the aim of universal zero-shot navigation, we propose a uniform graph
  representation to unify different goals, including object category, instance
  image and text description. We also convert the observation of agent into an
  online maintained scene graph. With this consistent scene and goal
  representation, we preserve most structural information compared with pure text
  and are able to leverage LLM for explicit graph-based reasoning. Specifically,
  we conduct graph matching between the scene graph and goal graph at each time
  instant and propose different strategies to generate long-term goal of
  exploration according to different matching states. The agent first iteratively
  searches subgraph of goal when zero-matched. With partial matching, the agent
  then utilizes coordinate projection and anchor pair alignment to infer the goal
  location. Finally scene graph correction and goal verification are applied for
  perfect matching. We also present a blacklist mechanism to enable robust switch
  between stages. Extensive experiments on several benchmarks show that our
  UniGoal achieves state-of-the-art zero-shot performance on three studied
  navigation tasks with a single model, even outperforming task-specific
  zero-shot methods and supervised universal methods.
paper_ai_keywords: universal zero-shot goal-oriented navigation zero-shot methods large language models (LLM) uniform graph representation object category instance image text description scene graph explicit graph-based reasoning graph matching long-term goal of exploration subgraph search coordinate projection anchor pair alignment scene graph correction goal verification blacklist mechanism UniGoal