Sensor fusion is a key technology that integrates various sensory inputs to allow for robust decision making in many applications such as autonomous driving and robot control. Deep neural networks have been adopted for sensor fusion in a body of recent studies. Among these, the so-called netgated architecture was proposed, which has demonstrated improved performances over the conventional convolutional neural networks (CNN). In this paper, we address several limitations of the baseline negated architecture by proposing two further optimized architectures: a coarser-grained gated architecture employing (feature) group-level fusion weights and a two-stage gated architectures leveraging both the group-level and featurelevel fusion weights. Using driving mode prediction and human activity recognition datasets, we demonstrate the significant performance improvements brought by the proposed gated architectures and also their robustness in the presence of sensor noise and failures.
This paper tackles the problem of sensor fusion, where multiple (possibly differing) sensor modalities are available and neural network architectures are used to combine information from them to perform prediction tasks. The paper proposed modifications to a gated fusion network specifically: 1) Grouping sets of sensors and concatenating them before further processing, and 2) Performing multi-level fusion where early sensor data representations are concatenated to produce weightings additional to the those obtained from features concatenated at a later stage. Experimental results show that these architectures achieve performance gains from 2-6%, especially when sensors are noisy or missing. 
Strengths
 + The architectures encourage fusion at multiple levels (especially the second one), which is a concept that has been successful across the deep learning literature
 
 + The paper looks at an interesting topic, especially related to looking at the effects of noise and missing sensors on the gating mechanisms. 
 + The results show some positive performance gains, although see caveats below. 
Weaknesses
 
 - The related work paragraph is extremely sparse. Fusion is an enormous field (see survey cited in this paper as well [1]), and I find the small choice of fusion results with a YouBot to be strange. A strong set of related work is necessary, focusing on those that are similar to the work. As an example spatiotemporal fusion (slow fusion [2]) bears some resemblence to this work but there are many others (e.g. [3,4] as a few examples).
 
   [1] Ramachandram, Dhanesh, and Graham W. Taylor. "Deep multimodal learning: A survey on recent advances and trends." IEEE Signal Processing Magazine 34.6 (2017): 96-108.
 	   Ramach
   [2] Karpathy, Andrej, et al. "Large-scale video classification with convolutional neural networks." Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2014
   [3] Mees, Oier, Andreas Eitel, and Wolfram Burgard. "Choosing smartly: Adaptive multimodal fusion for object detection in changing environments." Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 2016.
   [4] Kim, J., Koh, J., Kim, Y., Choi, J., Hwang, Y., & Choi, J. W. (2018). Robust Deep Multi-modal Learning Based on Gated Information Fusion Network. arXiv preprint arXiv:1807.06233.
 - The paper claims to provide a "deep understanding of the relationships between sensory inputs, fusion weights, network architecture, and resulting performance". I don't think it really achieves   
    this with the small examples of weights for some simple situations.
 - It is very unclear whether the architectures have more or less parameters. At one point it is stated that the original architecture overfits and the new architecture has less parameters (Sec 2.2 and 3). But then it is stated for fairness the number of neurons is equalized (5.2), and later in that section that the new architectures have additional neurons. Which of these is accurate?
 - Related to the previous point, and possibly the biggest weakness, the experimental methodology makes it hard to tell if performance is actually improved. For example, it is not clear to me that the performance gains are not just a result of less overfitting (for whatever reason) of baselines and that the fixed number of epochs therefore results in stopping at a better performance. Please show training and validation curves so that we can see whether the epochs chosen for the baselines are not just chosen after overfitting (in which case early stopping will improve the performance). As another example, there are no variances shown in the bar graphs. 
 - The examples with noise and failures are limited. For example, it is also not clear why an increase of noise in the RPM feature (Table 5) actually increases the weight of that group in the two-stage architecture. What does that mean? In general there isn't any principled method proposed for analyzing these situations. 
Some minor comments/clarifications:
  - What is the difference between these gated networks and attentional mechanisms, e.g. alpha attention (see "Attention is all you need" paper)?
  - What is a principled method to decide on the groupings?
  - There are several typos throughout the paper
    * "in the presence of snesor" => "in the presence of sensor"
    * Throughout the paper: "Predication" => "Prediction"
    * "Likelihood of stucking the training"
  - Tensorflow is not a simulation environment
Overall, the paper proposes architectural changes to an existing method for fusion, and while positive results are demonstrated there are several issues in the experimental methodology that make it unclear where the benefits come from. Further, the paper lacks novelty as multi-level fusion has been explored significantly and the changes are rather minor. There is no principled method or concepts that drive the architectural changes, and while the authors claim a deeper investigation into the networks' effectiveness under noise and failures the actual analysis is too shallow.