[Feature Request] Add 3D attention mask for T5Model
## Environment info

- `transformers` version: 4.2.1
- Platform: Linux
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0+cuda11.0

### Who can help
T5: @patrickvonplaten

## Information
The `get_extended_attention_mask()` does not sufficiently address the case with 3D attention mask. The problem emerges for T5Model as `input_ids` and `decoder_input_ids` are of different length and the `attention_mask` is of shape [Batch_size, Seq_length, Seq_length]. The decoder uses `attention_mask` directly for `encoder_attention_mask` in cross-attention, which is of incorrect shape and the error message does give any information about why it happens. 

## To reproduce
As described above. I can add code later if needed. 

## Expected behavior
I propose to add a sanity check for attention masks or improve the `get_extended_attention_mask()` method. 

Hello @yongyi-wu, 

yes you're right. T5 is not yet fully compatible with 3D attention_mask input. Currently, I won't find enough time to work on adding this feature, but I'll post it under "Community projects" in case someone from the community is interested in giving it a shot.

Also feel free to open a PR yourself, if you want to try :-) <|||||>Hi, I am new here but would like to give this a shot. Because it is my first issue I could use some direction if on how to tackle this, if this is okay with you. 

- Would you prefer the sanity check or an improved `get_extended_attention_mask()` method? 
- Do you know of an already existing implementations with 3D attention mask as a reference. 
- Where would you like to see the solution implemented. <|||||>Hi, I'm a newbie in transformers and trying to make customized 3D attention mask with T5ConditionalGeneration.
I was googling 3D attention mask for T5 model and found it in here.

From what I understood above is that, I should add one more `encoder_sequence_length` variable at the end of line `encoder_hidden_shape = (encoder_batch_size, encoder_sequence_length)` in T5Stack forward function if I want to build 3D attention?

Do I need to edit anything else?

@lexhuismans 

Thanks!