ACTIVE NEURAL LOCALIZATION
Localization is the problem of estimating the location of an autonomous agent from an observation and a map of the environment. Traditional methods of localization, which filter the belief based on the observations, are sub-optimal in the number of steps required, as they do not decide the actions taken by the agent. We propose "Active Neural Localizer", a fully differentiable neural network that learns to localize accurately and efficiently. The proposed model incorporates ideas of traditional filtering-based localization methods, by using a structured belief of the state with multiplicative interactions to propagate belief, and combines it with a policy model to localize accurately while minimizing the number of steps required for localization. Active Neural Localizer is trained end-to-end with reinforcement learning. We use a variety of simulation environments for our experiments which include random 2D mazes, random mazes in the Doom game engine and a photo-realistic environment in the Unreal game engine. The results on the 2D environments show the effectiveness of the learned policy in an idealistic setting while results on the 3D environments demonstrate the model's capability of learning the policy and perceptual model jointly from raw-pixel based RGB observations. We also show that a model trained on random textures in the Doom environment generalizes well to a photo-realistic office space environment in the Unreal engine.
The paper describes a neural network-based approach to active localization based upon RGB images. The framework employs Bayesian filtering to maintain an estimate of the agent's pose using a convolutional network model for the measurement (perception) function. A convolutional network models the policy that governs the action of the agent. The architecture is trained in an end-to-end manner via reinforcement learning. The architecture is evaluated in 2D and 3D simulated environments of varying complexity and compared favorably to traditional (structured) approaches to passive and active localization.
As the paper correctly points out, there is large body of work on map-based localization, but relatively little attention has been paid to decision theoretic formulations to localization, whereby the agent's actions are chosen in order to improve localization accuracy. More recent work instead focuses on the higher level objective of navigation, whereby any effort act in an effort to improve localization are secondary to the navigation objective. The idea of incorporating learned representations with a structured Bayesian filtering approach is interesting, but it's utility could be better motivated. What are the practical benefits to learning the measurement and policy model beyond (i) the temptation to apply neural networks to this problem and (ii) the ability to learn these in an end-to-end fashion? That's not to say that there aren't benefits, but rather that they aren't clearly demonstrated here. Further, the paper seems to assume (as noted below) that there is no measurement uncertainty and, with the exception of the 3D evaluations, no process noise.
The evaluation demonstrates that the proposed method yields estimates that are more accurate according to the proposed metric than the baseline methods, with a significant reduction in computational cost. However, the environments considered are rather small by today's standards and the baseline methods almost 20 years old. Further, the evaluation makes a number of simplifying assumptions, the largest being that the measurements are not subject to noise (the only noise that is present is in the motion for the 3D experiments). This assumption is clearly not valid in practice. Further, it is not clear from the evaluation whether the resulting distribution that is maintained is consistent (e.g., are the estimates over-/under-confident?). This has important implications if the system were to actually be used on a physical system. Further, while the computational requirements at test time are significantly lower than the baselines, the time required for training is likely very large. While this is less of an issue in simulation, it is important for physical deployments. Ideally, the paper would demonstrate performance when transferring a policy trained in simulation to a physical environment (e.g., using diversification, which has proven effective at simulation-to-real transfer).
Comments/Questions:
* The nature of the observation space is not clear.
* Recent related work has focused on learning neural policies for navigation, and any localization-specific actions are secondary to the objective of reaching the goal. It would be interesting to discuss how one would balance the advantages of choosing actions that improve localization with those in the context of a higher-level task (or at least including a cost on actions as with the baseline method of Fox et al.).
* The evaluation that assigns different textures to each wall is unrealistic.
* It is not clear why the space over which the belief is maintained flips as the robot turns and shifts as it moves.
* The 3D evaluation states that a 360 deg view is available. What happens when the agent can only see in one (forward) direction?
* AML includes a cost term in the objective. Did the author(s) experiment with setting this cost to zero?
* The 3D environments rely upon a particular belief size (70 x 70) being suitable for all environments. What would happen if the test environment was larger than those encountered in training?
* The comment that the PoseNet and VidLoc methods "lack a strainghtforward method to utilize past map data to do localization in a new environment" is unclear.
* The environments that are considered are quite small compared to the domains currently considered for
* Minor: It might be better to move Section 3 into Section 4 after introducing notation (to avoid redundancy).
* The paper should be proofread for grammatical errors (e.g., "bayesian" --> "Bayesian", "gaussian" --> "Gaussian")
UPDATES FOLLOWING AUTHORS' RESPONSE
(Apologies if this is a duplicate. I added a comment in light of the authors' response, but don't see it and so I am updating my review for completeness).
I appreciate the authors's response to the initial reviews and thank them for addressing several of my comments.
RE: Consistency
My concerns regarding consistency remain. For principled ways of evaluating the consistency of an estimator, see Bar-Shalom "Estimation with Applications to Tracking and Navigation".
RE: Measurement/Process Noise
The fact that the method assumes perfect measurements and, with the exception of the 3D experiments, no process noise is concerning as neither assumptions are valid for physical systems. Indeed, it is this noise in particular that makes localization (and its variants) challenging.
RE: Motivation
The response didn't address my comments about the lack motivation for the proposed method. Is it largely the temptation of applying an end-to-end neural method to a new problem? The paper should be updated to make the advantages over traditional approaches to active localization.