Kaiser, and Illia Polosukhin. 2017. Attention is all
you need. In Advances in neural information processing systems , pages 5998–6008.
Jesse Vig. 2019. Visualizing attention in transformerbased language models. arXiv preprint
arXiv:1904.02679 .
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018.
GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP , pages 353–355, Brussels, Belgium.
Association for Computational Linguistics.
Yequan Wang, Minlie Huang, Li Zhao, et al. 2016.
Attention-based lstm for aspect-level sentiment classiﬁcation. In proceedings of the 2016 conference on
empirical methods in natural language processing ,
pages 606–615.
Sarah Wiegreffe and Yuval Pinter. 2019. Attention is
not not explanation. In proceedings of the 2019 Conference on Empirical Methods in Natural Language
Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP) . Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien
In the Transformer model, "self-attention" combines information from attended
embeddings into the representation of the focal embedding in the next layer.
Thus, across layers of the Transformer, information originating from different
tokens gets increasingly mixed. This makes attention weights unreliable as
explanations probes. In this paper, we consider the problem of quantifying this
flow of information through self-attention. We propose two methods for
approximating the attention to input tokens given attention weights, attention
rollout and attention flow, as post hoc methods when we use attention weights
as the relative relevance of the input tokens. We show that these methods give
complementary views on the flow of information, and compared to raw attention,
both yield higher correlations with importance scores of input tokens obtained
using an ablation method and input gradients.
Samira Abnar Willem Zuidema