input attention we set j= 0.
Attention ﬂow In graph theory, a ﬂow network
is a directed graph with a “capacity” associated
with each edge. Formally, given G= (V; E)is a
graph, where Vis the set of nodes, and Eis the set
of edges in G;C=fcuv2Rj8u; vwhere eu;v2
E^u6=vgdenotes the capacities of the edges and
s; t2Vare the source and target (sink) nodes respectively; ﬂow is a mapping of edges to real numbers,f:E!R, that satisﬁes two conditions: (a)
capacity constraint : for each edge the ﬂow value
should not exceed its capacity, jfuvcuvj; (b)
ﬂow conservation : for all nodes except sandtthe
input ﬂow should be equal to output ﬂow –sum
of the ﬂow of outgoing edges should be equal to
sum of the ﬂow of incoming edges. Given a ﬂow
network, a maximum ﬂow algorithm ﬁnds a ﬂow
which has the maximum possible value between s
andt(Cormen et al., 2009).
In the Transformer model, "self-attention" combines information from attended
embeddings into the representation of the focal embedding in the next layer.
Thus, across layers of the Transformer, information originating from different
tokens gets increasingly mixed. This makes attention weights unreliable as
explanations probes. In this paper, we consider the problem of quantifying this
flow of information through self-attention. We propose two methods for
approximating the attention to input tokens given attention weights, attention
rollout and attention flow, as post hoc methods when we use attention weights
as the relative relevance of the input tokens. We show that these methods give
complementary views on the flow of information, and compared to raw attention,
both yield higher correlations with importance scores of input tokens obtained
using an ablation method and input gradients.
Samira Abnar Willem Zuidema