Attention Mechanism, Neural Networks, Artificial Intelligence, Machine Learning, Deep Learning.
which describes the input behaviour. We show such a decomposition aids in understanding the behaviour of networks with attention. In the tasks studied, we show the temporal (input) components play a larger role in determining the attention matrix as the average attention matrix becomes a better