Thanks for your reply! Yes, I understand in the encoder-decoder attention, attn_mask is memory_mask and key_pad_mask is memory_key_padding_mask, but I didnt quite get how these two mask looks like and how they work. Also, I am confused are theses two masks mandatory  when we use nn.TransformerDecoder? (In the pytorch documentation, it shows optional). I am also wondering when inference process, are these masks needed?