We present a soft, spatial, sequential, top-down attention model (S3TA). This model uses a soft attention mechanism to bottleneck its view of the input. A recurrent core is used to generate query vectors, which actively select information from the input by correlating the query with input-and space-dependent key maps at different spatial locations. We demonstrate the power and interpretabilty of this model under two settings. First, we build an agent which uses this attention model in RL environments and show that we can achieve performance competitive with state-of-the-art models while producing attention maps that elucidate some of the strategies used to solve the task. Second, we use this model in supervised learning tasks and show that it also achieves competitive performance and provides interpretable attention maps that show some of the underlying logic in the model's decision making.
[Summary]
This paper proposed a soft, spatial, sequential, top-down attention model which enable the agent and classifier actively select important, task relative information to generate the appropriate output. Given the observations, the proposed method uses a convLSTM to produce the key and value tensor. Different from multi-head attention, the query vector is generated in a top-down fashion. The authors proposed to augment the spatial feature with Fourier bases which is similar to previous work. The authors verify the proposed method on both reinforcement learning and supervised learning. On reinforcement learning, the proposed method outperformed the feedforward baseline and LSTM baseline. On reinforcement learning task, the proposed method achieves compelling result with more interpretable attention map that shows the model's decision. 
[Strength]
1: The proposed model is a straightforward extension of the multi-head attention to visual input. Compare to multi-head attention, it generates the query vector in a top-down manner instead of pure bottom up, and the authors verify the proposed choice is better than LSTM baseline empirically.
2: The authors verify the proposed method by extensive experiments on reinforcement learning tasks and also try supervised learning tasks. The attention visualization and human normalized scores for experts on ATARI show the effectiveness of the proposed method. 
[Weakness]
1: The soft spatial top-down attention is very common in vision and language domain, such as VQA. As the authors mentioned, the proposed method is very similar with MAC for CLEVER. The sequential attention is also explored in previous VQA work. Thus the novelty of the proposed method is limited. 
2: Multi-head attention for NLP tasks are usually composed with multiple layers. Will more layer of attention help the performance? The paper is less of this ablation study. 
3: The proposed method is worse compared with other baselines on supervised learning tasks, on both imagenet classification and kinetics. I wonder whether the recurrent process is required for those tasks? On table 2, we can observe that with sequence length 8, the performance is much worse,  this may be caused by overfitting. 
4: If the recurrent attention is more interpretable, given other visualization methods, such as gradcam, I wonder what is advantage?
5: I would expect that the performance on Kinetics dataset is better since sequential attention is required on video dataset. However, the performance is much worse compare of the baseline in the dataset. I wonder what is the reason? is there ablation study or any other results on this dataset?