Recent work on encoder-decoder models for sequence-to-sequence mapping has shown that integrating both temporal and spatial attentional mechanisms into neural networks increases the performance of the system substantially. We report on a new modular network architecture that applies an attentional mechanism not on temporal and spatial regions of the input, but on sensor selection for multi-sensor setups. This network called the sensor transformation attention network (STAN) is evaluated in scenarios which include the presence of natural noise or synthetic dynamic noise. We demonstrate how the attentional signal responds dynamically to changing noise levels and sensor-specific noise, leading to reduced word error rates (WERs) on both audio and visual tasks using TIDIGITS and GRID; and also on CHiME-3, a multi-microphone real-world noisy dataset. The improvement grows as more channels are corrupted as demonstrated on the CHiME-3 dataset. Moreover, the proposed STAN architecture naturally introduces a number of advantages including ease of removing sensors from existing architectures, attentional interpretability, and increased robustness to a variety of noise environments.
Summary: 
The authors consider the use of attention for sensor, or channel, selection. The idea is tested on several speech recognition datasets, including TIDIGITS and CHiME3, where the attention is over audio channels, and GRID, where the attention is over video channels. Results on TIDIGITS and GRID show a clear benefit of attention (called STAN here) over concatenation of features. The results on CHiME3 show gain over the CHiME3 baseline in channel-corrupted data.
Review:
The paper reads well, but as a standard application of attention lacks novelty. The authors mention that related work is generalized but fail to differentiate their work relative to even the cited references (Kim & Lane, 2016; Hori et al., 2017). Furthermore, while their approach is sold as a general sensor fusion technique, most of their experimentation is on microphone arrays with attention directly over magnitude-based input features, which cannot utilize the most important feature for signal separation using microphone arrays---signal phase. Their results on CHiME3 are terrible: the baseline CHiME3 system is very weak, and their system is only slightly better! The winning system has a WER of only 5.8%(vs. 33.4% for the baseline system), while more than half of the submissions to the challenge were able to cut the WER of the baseline system in half or better! http://spandh.dcs.shef.ac.uk/chime_challenge/chime2015/results.html. Their results wrt channel corruption on CHiME3, on the other hand, are reasonable, because the model matches the problem being addressed…
Overall Assessment: 
In summary, the paper lacks novelty wrt technique, and as an “application-of-attention” paper fails to be even close to competitive with the state-of-the-art approaches on the problems being addressed. As such, I recommend that the paper be rejected.
Additional comments: 
-	The experiments in general lack sufficient detail: Were the attention masks trained supervised or unsupervised? Were the baselines with concatenated features optimized independently? Why is there no multi-channel baseline for the GRID results? 
-	Issue with noise bursts plot (Input 1+2 attention does not sum to 1)
-	A concatenation based model can handle a variable #inputs: it just needs to be trained/normalized properly during test (i.e. like dropout)…