Published as a conference paper at ICLR 2018 MEMORY AUGMENTED CONTROL NETWORKS
Planning problems in partially observable environments cannot be solved directly with convolutional networks and require some form of memory. But, even memory networks with sophisticated addressing schemes are unable to learn intelligent reasoning satisfactorily due to the complexity of simultaneously learning to access memory and plan. To mitigate these challenges we propose the Memory Augmented Control Network (MACN). The network splits planning into a hierarchical process. At a lower level, it learns to plan in a locally observed space. At a higher level, it uses a collection of policies computed on locally observed spaces to learn an optimal plan in the global environment it is operating in. The performance of the network is evaluated on path planning tasks in environments in the presence of simple and complex obstacles and in addition, is tested for its ability to generalize to new environments not seen in the training set.
The paper presents a method for navigating in an unknown and partially observed environment is presented. The proposed approach splits planning into two levels: 1) local planning based on the observed space and 2) a global planner which receives the local plan, observation features, and access to an addressable memory to decide on which action to select and what to write into memory. 
The contribution of this work is the use of value iteration networks (VINs) for local planning on a locally observed map that is fed into a learned global controller that references history and a differential neural computer (DNC), local policy, and observation features select an action and update the memory. The core concept of learned local planner providing additional cues for a global, memory-based planner is a clever idea and the thorough analysis clearly demonstrates the benefit of the approach.
The proposed method is tested against three problems: a gridworld, a graph search, and a robot environment. In each case the proposed method is more performant than the baseline methods.  The ablation study of using LSTM instead of the DNC and the direct comparison of CNN + LSTM support the authors’ hypothesis about the benefits of the two components of their method. While the author’s compare to DRL methods with limited horizon (length 4), there is no comparison to memory-based RL techniques. Furthermore, a comparison of related memory-based visual navigation techniques on domains for which they are applicable should be considered as such an analysis would illuminate the relative performance over the overlapping portions problem domains  For example, analysis of the metric map approaches on the grid world or of MACN on their tested environments.
Prior work in visual navigation in partially observed and unknown environments have used addressable memory (e.g., Oh et al.) and used VINs (e.g., Gupta et al.) to plan as noted. In discussing these methods, the authors state that these works are not comparable as they operate strictly on discretized 2d spaces. However, it appears to the reviewer that several of these methods can be adapted to higher dimensions and be applicable at least a subclass (for the euclidean/metric map approaches) or the full class of the problems (for Oh et al.), which appears to be capable to solve non-euclidean tasks like the graph search problem. If this assessment is correct, the authors should differentiate between these approaches more thoroughly and consider empirical comparisons. The authors should further consider contrasting their approach with “Neural SLAM” by Zhang et al.
A limitation of the presented method is requirement that the observation “reveals the labeling of nearby states.” This assumption holds in each of the examples presented: the neighborhood map in the gridworld and graph examples and the lidar sensor in the robot navigation example. It would be informative for the authors to highlight this limitation and/or identify how to adapt the proposed method under weaker assumptions such as a sensor that doesn’t provide direct metric or connectivity information such as a RGB camera. 
Many details of the paper are missing and should be included to clarify the approach and ensure reproducible results. The reviewer suggests providing both more details in the main section of the paper and providing the precise architecture including hyperparameters in the supplementary materials section.