LiHaibo:

src_mask


@LiHaibo
First, both masks work on the dot product of query and key in the “Scaled Dot-Product Attention” layer.
src_mask is working on the matrix with a dimension of (S, S) and add ‘-inf’ to a single position. src_key_padding_mask is more like a padding marker, which masks a specific tokens in the src sequence (a.k.a. the entire column/row of the attention matrix is set to ‘-inf’).