SEMI-PARAMETRIC TOPOLOGICAL MEMORY FOR NAVIGATION
We introduce a new memory architecture for navigation in previously unseen environments, inspired by landmark-based navigation in animals. The proposed semiparametric topological memory (SPTM) consists of a (non-parametric) graph with nodes corresponding to locations in the environment and a (parametric) deep network capable of retrieving nodes from the graph based on observations. The graph stores no metric information, only connectivity of locations corresponding to the nodes. We use SPTM as a planning module in a navigation system. Given only 5 minutes of footage of a previously unseen maze, an SPTM-based navigation agent can build a topological map of the environment and use it to confidently navigate towards goals. The average success rate of the SPTM agent in goal-directed navigation across test environments is higher than the best-performing baseline by a factor of three.
*** Revision: based on the author's work, we have switched the score to accept (7) ***
Clever ideas but not end-to-end navigation.
This paper presents a hybrid architecture that mixes parametric (neural) and non-parametric (Dijkstra's path planning on a graph of image embeddings) elements and applies it to navigation in unseen 3D environments (Doom). The path planning in unseen environments is done in the following way: first a human operator traverses the entire environment by controlling the agent and collecting a long episode of 10k frames that are put into a chain graph. Then loop closures are automatically detected using image similarity in feature space, using a localization feed-forward ResNet (trained using a DrLIM-like triplet loss on time-similar images), resulting in a topological graph where edges correspond to similar viewpoints or similar time points. For a given target position image and agent start position image, a nearest neighbor search-powered Dijkstra path planning is done on the graph to create a list of waypoint images. The pairs of (current image, next waypoint) images are then fed to a feed-forward locomotion (policy) network, trained in a supervised manner.
The paper does not discuss at all the problems arising when the images are ambiguous: since the localisation network is feed-forward, surely there must be images that are ambiguously mapped to different graph areas and are closing loops erroneously? The problem is mitigated by the fact that a human operator controls the agent, making sure that the agent's viewpoint is clear, but the method will probably fail if the agent is learning to explore the maze autonomously, bumping into walls and facing walls. The screenshots on Figure 3 suggest that the walls have a large variety of textures and decorations, making each viewpoint potentially unique, unlike the environments in (Mirowski et al, 2017), (Jaderberg et al, 2017) and (Mnih et al, 2016).
Most importantly, the navigation is not based on RL at all, and ignores the problem of exploration altogether. A human operator labels 10k frames by playing the game and controlling the agent, to show it how the maze looks like and what are the paths to be taken. As a consequence, comparison to end-to-end RL navigation methods is unclear, and should be stressed upon in the manuscript. This is NOT a proper navigation agent.
Additional baselines should be evaluated: 1) a fully Dijkstra-based baseline where the direction of motion of the agent along the edge is retrieved and used to guide the agent (i.e., the policy becomes a lookup table on image pairs) and 2) the same but the localization network replaced by image similarities in pixel space or some image descriptor space (e.g., SURF, ORB, etc…). It seems to me that those baselines would be very strong.
Another baseline is missing: (Oh et al, 2016, “Control of Memory, Active Perception, and Action in Minecraft”).
The paper is not without merit: the idea of storing experiences in a graph and in using landmark similarity rather than metric embeddings is interesting. Unfortunately, that episodic memory is not learned (e.g., Neural Turing Machines or Memory Networks).
In summary, just like the early paper released in 2010 about Kinect-based RGBD SLAM: lots of excitement but potential disappointment when the method is applied on an actual mobile robot, navigating in normal environments with visual ambiguity and white walls. The paper should ultimately be accepted to this conference to provide a baseline for the community  (once the claims are revised), but I street that the claims of learning to navigate in unseen environments are unsubstantiated, as the method is neither end-to-end learned (as it relies on human input and heuristic path planning) nor capable of exploring unseen environments with visual ambiguity.