Algorithms, Third Edition , 3rd edition. The MIT
Press.
Mostafa Dehghani, Stephan Gouws, Oriol Vinyals,
Jakob Uszkoreit, and Łukasz Kaiser. 2019. Universal transformers. In proceedings of the 2019 International Conference on Learning Representations .
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and
Kristina Toutanova. 2019. BERT: Pre-training of
deep bidirectional transformers for language understanding. In proceedings of the 2019 Conference
of the North American Chapter of the Association
for Computational Linguistics: Human Language
Technologies . Association for Computational Linguistics.
Sarthak Jain and Byron C. Wallace. 2019. Attention
is not Explanation. In proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 3543–3556. Association
for Computational Linguistics.
Jaesong Lee, Joong-Hwi Shin, and Jun-Seok Kim.
2017. Interactive visualization and manipulation
of attention-based neural machine translation. In
proceedings of the 2017 Conference on Empirical
Methods in Natural Language Processing: System
Demonstrations , pages 121–126.
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg.
In the Transformer model, "self-attention" combines information from attended
embeddings into the representation of the focal embedding in the next layer.
Thus, across layers of the Transformer, information originating from different
tokens gets increasingly mixed. This makes attention weights unreliable as
explanations probes. In this paper, we consider the problem of quantifying this
flow of information through self-attention. We propose two methods for
approximating the attention to input tokens given attention weights, attention
rollout and attention flow, as post hoc methods when we use attention weights
as the relative relevance of the input tokens. We show that these methods give
complementary views on the flow of information, and compared to raw attention,
both yield higher correlations with importance scores of input tokens obtained
using an ablation method and input gradients.
Samira Abnar Willem Zuidema