In many robotic applications, it is crucial to maintain a belief about the state of a system, like the location of a robot or the pose of an object. These state estimates serve as input for planning and decision making and provide feedback during task execution. Recursive Bayesian Filtering algorithms address the state estimation problem, but they require a model of the process dynamics and the sensory observations as well as noise estimates that quantify the accuracy of these models. Recently, multiple works have demonstrated that the process and sensor models can be learned by end-to-end training through differentiable versions of Recursive Filtering methods. However, even if the predictive models are known, finding suitable noise models remains challenging. Therefore, many practical applications rely on very simplistic noise models. Our hypothesis is that end-to-end training through differentiable Bayesian Filters enables us to learn more complex heteroscedastic noise models for the system dynamics. We evaluate learning such models with different types of filtering algorithms and on two different robotic tasks. Our experiments show that especially for sampling-based filters like the Particle Filter, learning heteroscedastic noise models can drastically improve the tracking performance in comparison to using constant noise models.
This is a well written paper which proposes to learn heteroscedastic noise models from data by optimizing the prediction likelihood end-to-end through differentiable Bayesian Filters. In addition to existing Bayesian filters, the paper also proposes two different versions of the [differentiable] Unscented Kalman Filter. Performance of the different filters and noise models is evaluated on two real-world robotic problems: Visual Odometry and visual tracking of an object pushed by the robot.
While the general idea of learning the noise variances through backpropagation are straightforward extensions of existing work on differential Bayesian filters, the questions that the paper explores are important to make end-to-end learning of Bayesian filter more common. The results will help future research select the correct differential filter for their use case, and insight in potential benefits (or lack thereof) by learning heteroscedastic or homoscedastic process noise, and/or observation noise.
A downside is that the paper does not further explore how to weigh different loss terms which are apparently important to successfully train such models. Also unfortunate is the footnote which states that the current results are incomplete and will be updated, hence as a reviewer I am not sure which results and conclusions are valid right now.
Pros:
+ clearly written
+ useful experiments for those seeking to select a differential Bayesian filter, and learning (heteroscedastic) noise from data.
+ experiments on real-world use cases rather than toy problems
Cons:
- Incomplete experiments according to footnote, thus results and conclusions might change after this review.
- Unclear what the effect of the selected process / observation model is on the learned noise
Below are more detailed comments and questions:
* p6. Footnote: "due to time constraints, ..., results will be updated" Is this acceptable? I have never seen such a notice when reviewing. So, are the current results on a single fold? Will the numbers in the tables, or the conclusions change after this review?
* If I understand correctly, the paper 'only' focuses on learning the heteroscedastic noise variance, but assumes that the deterministic non-linear parts of the process and observation models are fixed. I did not find this very clearly stated in the paper, though at least the Appendix explicitly states the used functions for the process models.
* I would have liked to see in the paper more explanation on how the process and observations models were selected and validated  in the experiments, since I expect that the validity of these functions affects the learned noise variances. Since the noise needs to account for the inaccuracies in the deterministic models, would the choice for these functions not impact your conclusions? And, would it or would it not be possible to learn both these deterministic models and the noise jointly from the training data?
* Is it possible to add priors on Q and R parameters for Bayesian treatment of learning model parameters? I can imagine that priors can guide the optimization to either adjust more of the Q or more of the R variance to improve the likelihood.
* Section 1:
	* "Our experiments show that ... " This may be a matter of taste, but I did not expect to see the main conclusions already in the introduction. They should appear in the abstract to help out the quick reader. In the introduction, it appears as if you are talking about some separate preliminary experiments, and which you base some conclusions that will be used in the remainder of this paper.
* Section 3:
	* So, mostly empirical study, since heteroscedastic noise models were already used?
	* "Previous work evaluated ... " please add citations
* Section 4.1:
	* "train a discriminative neural network o with parameters wo to preprocess the raw sensory data D and thus create a more compact representation of the observations z = o(D;wo)." At this point in the paper, I don't understand this. How is z learned, via supervised learning (what is the target value for z)? Or is z some latent representation that is jointly optimized with the filters? This only became somewhat clearer in Sec. 5.2 on p.8 where it states that "We ... train a neural network to extract the position of the object, the contact point and normal as well as ...". So if I understand correctly, the function o for z = o(D) is thus learned offline w.r.t. some designed observation variables for which GT is available (from manual annotations?).
* Section 4.2:
	* "we predict a separate Qi for every sigma point and then compute Q as the weighted mean" → So, separate parameters w_g for each sigma point i, or is a single learned non-linear function applied to all points?
* Section 4.3:
	* Equation 14: inconsistent use of boldface script: should use bold sigma_t, and bold l_t ?
	* "In practice, we found that during learning ... by only increasing the predicted variance" →  This is an interesting observation, which I would have liked to see explored more. I understand that term (ii) is needed to guide the learning processes, but in the end wouldn't we want to optimize the actual likelihood? So, could you (after the loss with (ii) converged) reduce \lambda_2 to zero to properly optimize only the log likelihood without guidance from a good initial state? Or is it not possible to reliably optimize the likelihood via back-propagation at all from some reason?
* Section 5.1.1
	* "... of varying length (from 270 to over 4500 steps) ..." it would be good to mention the fps, to get understand to what real-world time horizons 50 / 100 frames correspond.
* Section 5.1.2:
	* Table 1: How are the parameters of the filters in the "no learning" column obtained? Are these tuned in some other way, or taken form existing implementations? Also, can you clarify if the 'no learning' parameters served as the initial condition for the learning approaches?
	* Table 1, first row column Q+R: "0.2" → Is there a missing zero here, i.e. "0.20"? Otherwise, the precision of reported results in this table is not consistent. Hard to say: is the mean of R+Q 0.2, and slightly lower than R+Qh, or could it be as high as 0.24 ?
	* "learning a heteroscedastic process noise model leads to big improvements and makes the filters competitive with the EKF". Results for EKF still appear significantly better than the novel UKF, and even the PF (especially rotational error).
* Section 6: 
	* "Large outliers in the prediction of the preprocessing networks were not associated with higher observation noise." I don't see on what presented results these conclusions were drawn, as this is the first time the word "outlier" is mentioned in the paper. Outliers seem indeed important, as they contradict the typical assumptions e.g. of Gaussian noise, so it would be useful to clarify how the proposed techniques handle such outliers.