Thomas Wolf. 2019. Distilbert, a distilled version of
bert: smaller, faster, cheaper and lighter.
Soﬁa Serrano and Noah A. Smith. 2019. Is attention
interpretable? In proceedings of the 57th Annual
Meeting of the Association for Computational Linguistics . Association for Computational Linguistics.
Richard Socher, Alex Perelygin, Jean Wu, Jason
Chuang, Christopher D. Manning, Andrew Ng, and
Christopher Potts. 2013. Recursive deep models
for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on
Empirical Methods in Natural Language Processing ,
pages 1631–1642, Seattle, Washington, USA. Association for Computational Linguistics.
Shikhar Vashishth, Shyam Upadhyay, Gaurav Singh
Tomar, and Manaal Faruqui. 2019. Attention interpretability across nlp tasks. arXiv preprint
arXiv:1909.11218 .
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob
Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz
Kaiser, and Illia Polosukhin. 2017. Attention is all
you need. In Advances in neural information processing systems , pages 5998–6008.
In the Transformer model, "self-attention" combines information from attended
embeddings into the representation of the focal embedding in the next layer.
Thus, across layers of the Transformer, information originating from different
tokens gets increasingly mixed. This makes attention weights unreliable as
explanations probes. In this paper, we consider the problem of quantifying this
flow of information through self-attention. We propose two methods for
approximating the attention to input tokens given attention weights, attention
rollout and attention flow, as post hoc methods when we use attention weights
as the relative relevance of the input tokens. We show that these methods give
complementary views on the flow of information, and compared to raw attention,
both yield higher correlations with importance scores of input tokens obtained
using an ablation method and input gradients.
Samira Abnar Willem Zuidema