Thank you for your reply! For my understanding, there are two masks in the MultiHeadAttention, one is attn_mask and the other is key_pad_mask. In the decoder, the first Multi-head self-attention takes tgt_mask as attn_mask which is used to prevent decoder see its subsequent tokens, and key_pad_mak is the padding mask for target sequence. The second multi-head attention is contextual attention, which leverage the output from encoder and decoder self-attention. I don’t quite understand  attn_mask and key_pad_mask in the contextual attention, and wondering whether could you explain those two masks? Thank you!