They are optional as their default values are None.
It might be helpful for you to understand that there are three types of attention in the Transformer:



encoder self-attention (no mask needed)




encoder-decoder attention (no mask needed)




decoder self-attention (mask needed)



In some sense, attention is a way to calculate a vector representation of a token using its context. (2) encoder-decoder attention calculates a vector representation of each token in the (generated) output sequence based on all tokens in the input sequence. Thus, it’s called encoder-decoder attention.
Unless you don’t want to use some certain token(s) in the input sequence for some output token(s), there is no reason to mask the information from the input sequence. Therefore, the default mask is None.
The following article visually explains how encoder/decoder self-attention and encoder-decoder attention work in the Transformer.

  
      jalammar.github.io
  
  
    

The Illustrated Transformer 49

Discussions:
Hacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)


Translations: Chinese (Simplified), Korean

Watch: MIT’s Deep Learning State of the Art lecture referencing this post

In the previous post, we...