🐛 Bug in attention head mask for cross-attention module in encoder-decoder models
Currently, encoder-decoder models use either `head_mask` or `decoder_head_mask` for masking attention heads in cross-attention modules. Both cases are not perfectly correct. Furthermore, MHA in cross-attention modules shares the parameters with the decoder, i.e. `shape = (decoder.num_layers, decoder.num_attention_heads)`, therefore, the usage of encoder `head_mask` in the cross-attention module may lead to errors due to the shape mismatch.

<hr>

**My contribution:** I will take care of this issue this weekend.

<hr>

**Reviewers:** @patil-suraj @patrickvonplaten 


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>Unstale<|||||>Hey @stancld,

Sorry for being so unresponsive here - I'm happy to change the behavior and provide 3 masks