  Embodied artificial intelligence (AI) tasks shift from tasks focusing on
internet images to active settings involving embodied agents that perceive and
act within 3D environments. In this paper, we investigate the target-driven
visual navigation using deep reinforcement learning (DRL) in 3D indoor scenes,
whose navigation task aims to train an agent that can intelligently make a
series of decisions to arrive at a pre-specified target location from any
possible starting positions only based on egocentric views. However, most
navigation methods currently struggle against several challenging problems,
such as data efficiency, automatic obstacle avoidance, and generalization.
Generalization problem means that agent does not have the ability to transfer
navigation skills learned from previous experience to unseen targets and
scenes. To address these issues, we incorporate two designs into classic DRL
framework: attention on 3D knowledge graph (KG) and target skill extension
(TSE) module. On the one hand, our proposed method combines visual features and
3D spatial representations to learn navigation policy. On the other hand, TSE
module is used to generate sub-targets which allow agent to learn from
failures. Specifically, our 3D spatial relationships are encoded through
recently popular graph convolutional network (GCN). Considering the real world
settings, our work also considers open action and adds actionable targets into
conventional navigation situations. Those more difficult settings are applied
to test whether DRL agent really understand its task, navigating environment,
and can carry out reasoning. Our experiments, performed in the AI2-THOR, show
that our model outperforms the baselines in both SR and SPL metrics, and
improves generalization ability across targets and scenes.

Computer Vision and Pattern Recognition;Machine Learning;Machine Learning