Sensor Transformation Attention Networks
Recent work on encoder-decoder models for sequence-to-sequence mapping has shown that integrating both temporal and spatial attentional mechanisms into neural networks increases the performance of the system substantially. We report on a new modular network architecture that applies an attentional mechanism not on temporal and spatial regions of the input, but on sensor selection for multi-sensor setups. This network called the sensor transformation attention network (STAN) is evaluated in scenarios which include the presence of natural noise or synthetic dynamic noise. We demonstrate how the attentional signal responds dynamically to changing noise levels and sensor-specific noise, leading to reduced word error rates (WERs) on both audio and visual tasks using TIDIGITS and GRID; and also on CHiME-3, a multi-microphone real-world noisy dataset. The improvement grows as more channels are corrupted as demonstrated on the CHiME-3 dataset. Moreover, the proposed STAN architecture naturally introduces a number of advantages including ease of removing sensors from existing architectures, attentional interpretability, and increased robustness to a variety of noise environments.
 This paper proposes sensor transformation attention network (STAN), which dynamically select appropriate sequential sensor inputs based on an attention mechanism. Pros: One of the main focuses of this paper is to apply this method to a real task, multichannel speech recognition based on CHiME-3, by providing its reasonable sensor selection function in real data especially to avoid audio data corruptions. This analysis is quite intuitive, and also shows the effectiveness of the proposed method in this practical setup. Cons: The idea seems to be simple and does not have significant originality. Also, the paper does not clearly mention the attention mechanism part, and needs some improvement. Comments: - The paper mainly focuses on the soft sensor selection. However, in an array signal processing context (and its application to multichannel speech recognition), it would be better to mention beamforming techniques, where the compensation of the delays of sensors is quite important. - In addition, there is a related study of using multichannel speech recognition based on sequence-to-sequence modeling and attention mechanism by Ochiai et al, "A Unified Architecture for Multichannel End-to-End Speech Recognition with Neural Beamforming," IEEE Journal of Selected Topics in Signal Processing. This paper uses the same CHiME-3 database, and also showing a similar analysis of channel selection. It’s better to discuss about this paper as well as a reference. - Section 2: better to explain about how to obtain attention scores z in more details. - Figure 3, experiments of Double audio/video clean conditions: I cannot understand why they are improved from single audio/video clean conditions. Need some explanations. - Section 3.1: 39-dimensional Mel-frequency cepstral coefficients (MFCCs) -> 13 -dimensional Mel-frequency cepstral coefficients (MFCCs) with 1st and 2nd order delta features. - Section 3.2 Dataset “As for TIDIGIT”: “As for GRID”(?) - Section 4 Models “The parameters of the attention modules are either shared across sensors (STAN-shared) or not shared across sensors (STAN- default).”: It’s better to explain this part in more details, possibly with some equations. It is hard to understand the difference.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Sensor Transformation Attention Networks
Recent work on encoder-decoder models for sequence-to-sequence mapping has shown that integrating both temporal and spatial attentional mechanisms into neural networks increases the performance of the system substantially. We report on a new modular network architecture that applies an attentional mechanism not on temporal and spatial regions of the input, but on sensor selection for multi-sensor setups. This network called the sensor transformation attention network (STAN) is evaluated in scenarios which include the presence of natural noise or synthetic dynamic noise. We demonstrate how the attentional signal responds dynamically to changing noise levels and sensor-specific noise, leading to reduced word error rates (WERs) on both audio and visual tasks using TIDIGITS and GRID; and also on CHiME-3, a multi-microphone real-world noisy dataset. The improvement grows as more channels are corrupted as demonstrated on the CHiME-3 dataset. Moreover, the proposed STAN architecture naturally introduces a number of advantages including ease of removing sensors from existing architectures, attentional interpretability, and increased robustness to a variety of noise environments.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper proposes sensor transformation attention network (STAN), which dynamically select appropriate sequential sensor inputs based on an attention mechanism. Pros: One of the main focuses of this paper is to apply this method to a real task, multichannel speech recognition based on CHiME-3, by providing its reasonable sensor selection function in real data especially to avoid audio data corruptions. This analysis is quite intuitive, and also shows the effectiveness of the proposed method in this practical setup. Cons: The idea seems to be simple and does not have significant originality. Also, the paper does not clearly mention the attention mechanism part, and needs some improvement. Comments: - The paper mainly focuses on the soft sensor selection. However, in an array signal processing context (and its application to multichannel speech recognition), it would be better to mention beamforming techniques, where the compensation of the delays of sensors is quite important. - In addition, there is a related study of using multichannel speech recognition based on sequence-to-sequence modeling and attention mechanism by Ochiai et al, "A Unified Architecture for Multichannel End-to-End Speech Recognition with Neural Beamforming," IEEE Journal of Selected Topics in Signal Processing. This paper uses the same CHiME-3 database, and also showing a similar analysis of channel selection. It’s better to discuss about this paper as well as a reference. - Section 2: better to explain about how to obtain attention scores z in more details. - Figure 3, experiments of Double audio/video clean conditions: I cannot understand why they are improved from single audio/video clean conditions. Need some explanations. - Section 3.1: 39-dimensional Mel-frequency cepstral coefficients (MFCCs) -> 13 -dimensional Mel-frequency cepstral coefficients (MFCCs) with 1st and 2nd order delta features. - Section 3.2 Dataset “As for TIDIGIT”: “As for GRID”(?) - Section 4 Models “The parameters of the attention modules are either shared across sensors (STAN-shared) or not shared across sensors (STAN- default).”: It’s better to explain this part in more details, possibly with some equations. It is hard to understand the difference.
assistant
 This paper proposes sensor transformation attention network (STAN), which dynamically select appropriate sequential sensor inputs based on an attention mechanism. Pros: One of the main focuses of this paper is to apply this method to a real task, multichannel speech recognition based on CHiME-3, by providing its reasonable sensor selection function in real data especially to avoid audio data corruptions. This analysis is quite intuitive, and also shows the effectiveness of the proposed method in this practical setup. Cons: The idea seems to be simple and does not have significant originality. Also, the paper does not clearly mention the attention mechanism part, and needs some improvement. Comments: - The paper mainly focuses on the soft sensor selection. However, in an array signal processing context (and its application to multichannel speech recognition), it would be better to mention beamforming techniques, where the compensation of the delays of sensors is quite important. - In addition, there is a related study of using multichannel speech recognition based on sequence-to-sequence modeling and attention mechanism by Ochiai et al, "A Unified Architecture for Multichannel End-to-End Speech Recognition with Neural Beamforming," IEEE Journal of Selected Topics in Signal Processing. This paper uses the same CHiME-3 database, and also showing a similar analysis of channel selection. It’s better to discuss about this paper as well as a reference. - Section 2: better to explain about how to obtain attention scores z in more details. - Figure 3, experiments of Double audio/video clean conditions: I cannot understand why they are improved from single audio/video clean conditions. Need some explanations. - Section 3.1: 39-dimensional Mel-frequency cepstral coefficients (MFCCs) -> 13 -dimensional Mel-frequency cepstral coefficients (MFCCs) with 1st and 2nd order delta features. - Section 3.2 Dataset “As for TIDIGIT”: “As for GRID”(?) - Section 4 Models “The parameters of the attention modules are either shared across sensors (STAN-shared) or not shared across sensors (STAN- default).”: It’s better to explain this part in more details, possibly with some equations. It is hard to understand the difference.