title: SAME: Learning Generic Language-Guided Visual Navigation with
    State-Adaptive Mixture of Experts
summary: The academic field of learning instruction-guided visual navigation can be
  generally categorized into high-level category-specific search and low-level
  language-guided navigation, depending on the granularity of language
  instruction, in which the former emphasizes the exploration process, while the
  latter concentrates on following detailed textual commands. Despite the
  differing focuses of these tasks, the underlying requirements of interpreting
  instructions, comprehending the surroundings, and inferring action decisions
  remain consistent. This paper consolidates diverse navigation tasks into a
  unified and generic framework -- we investigate the core difficulties of
  sharing general knowledge and exploiting task-specific capabilities in learning
  navigation and propose a novel State-Adaptive Mixture of Experts (SAME) model
  that effectively enables an agent to infer decisions based on
  different-granularity language and dynamic observations. Powered by SAME, we
  present a versatile agent capable of addressing seven navigation tasks
  simultaneously that outperforms or achieves highly comparable performance to
  task-specific agents.
paper_authors._id: 675b9d8aa86e549855482d11
paper_authors.name: Gengze Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-16T09:42:27.297Z
paper_authors._id: 6450bcd3673b2bcfaf8681af
paper_authors.avatarUrl: /avatars/f5f93d780562d0772ec5dc1728945fcf.svg
paper_authors.fullname: Gengze Zhou
paper_authors.type: user
paper_authors.user: ZGZzz
paper_authors._id: 675b9d8aa86e549855482d12
paper_authors.name: Yicong Hong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T10:53:12.463Z
paper_authors._id: 654ca66fe1671abcbc50a6e4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/654ca66fe1671abcbc50a6e4/VqcnC17YcqC9Hk65_LxtE.jpeg
paper_authors.fullname: Yicong Hong
paper_authors.type: user
paper_authors.user: YicongHong
paper_authors._id: 675b9d8aa86e549855482d13
paper_authors.name: Zun Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T10:53:06.791Z
paper_authors._id: 6622c710b0e5c5e3de8311c1
paper_authors.avatarUrl: /avatars/a824c150040731679bbd77762ca9d4eb.svg
paper_authors.fullname: Zun Wang
paper_authors.type: user
paper_authors.user: ZunWang
paper_authors._id: 675b9d8aa86e549855482d14
paper_authors.name: Chongyang Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T10:53:01.021Z
paper_authors._id: 64c7671e77a6473ac325b6d8
paper_authors.avatarUrl: /avatars/65434ae55207e785bd1ff2798d612fc2.svg
paper_authors.fullname: Chongyang Zhao
paper_authors.type: user
paper_authors.user: zhaoc5
paper_authors._id: 675b9d8aa86e549855482d15
paper_authors.name: Mohit Bansal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T10:52:54.647Z
paper_authors._id: 665d9d3a057f7c508f98c625
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/665d9d3a057f7c508f98c625/u1R9P9sJoAl4zEIcetbPy.jpeg
paper_authors.fullname: Mohit Bansal
paper_authors.type: user
paper_authors.user: mohitbansal
paper_authors._id: 675b9d8aa86e549855482d16
paper_authors.name: Qi Wu
paper_title: SAME: Learning Generic Language-Guided Visual Navigation with
    State-Adaptive Mixture of Experts
paper_summary: The academic field of learning instruction-guided visual navigation can be
  generally categorized into high-level category-specific search and low-level
  language-guided navigation, depending on the granularity of language
  instruction, in which the former emphasizes the exploration process, while the
  latter concentrates on following detailed textual commands. Despite the
  differing focuses of these tasks, the underlying requirements of interpreting
  instructions, comprehending the surroundings, and inferring action decisions
  remain consistent. This paper consolidates diverse navigation tasks into a
  unified and generic framework -- we investigate the core difficulties of
  sharing general knowledge and exploiting task-specific capabilities in learning
  navigation and propose a novel State-Adaptive Mixture of Experts (SAME) model
  that effectively enables an agent to infer decisions based on
  different-granularity language and dynamic observations. Powered by SAME, we
  present a versatile agent capable of addressing seven navigation tasks
  simultaneously that outperforms or achieves highly comparable performance to
  task-specific agents.
paper_ai_keywords: State-Adaptive Mixture of Experts (SAME) language-guided navigation high-level category-specific search low-level language-guided navigation language instruction interpreting instructions comprehending the surroundings inferring action decisions sharing general knowledge exploiting task-specific capabilities learning navigation dynamic observations versatile agent navigation tasks