Relative Attention Bias not initialized for T5ForConditionalGeneration in version 4.0.0
## Environment info
     
- `transformers` version: 4.0.0
- Platform: Linux 18.04
- Python version: 3.6.9
- PyTorch version (GPU?): 1.70 (True)
- Tensorflow version (GPU?): N/A
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help

@patrickvonplaten

## Information

Model I am using: T5ForConditionalGeneration, specifically the "allenai/unifiedqa-t5-large" checkpoint from the model hub

The problem arises when I try to load the checkpoint following standard loading procedures under `transformers==4.0.0`. The same doesn't happen in version `3.5.0`.

## To reproduce

```python
from transformers import AutoTokenizer, T5ForConditionalGeneration
model_name_or_path = "allenai/unifiedqa-t5-large"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
# Warnings in version 4.0, not in 3.5.0 an preceding ones
model = T5ForConditionalGeneration.from_pretrained(model_name_or_path)
```
```
Some weights of the model checkpoint at allenai/unifiedqa-t5-large were not used when initializing T5ForConditionalGeneration: ['decoder.blo
ck.0.layer.1.EncDecAttention.relative_attention_bias.weight']                                                                               
- This IS expected if you are initializing T5ForConditionalGeneration from the checkpoint of a model trained on another task or with another
 architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).                                    
- This IS NOT expected if you are initializing T5ForConditionalGeneration from the checkpoint of a model that you expect to be exactly ident
ical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
```

## Expected behavior

A consistent behavior across versions, either always or never raising the warning at loading time.

Hey @gsarti, 

yeah there was a tiny bug in T5 previously. T5 actually never had relative positional encodings for the EncoderDecoderLayer, so the unnecessary weight was deleted after 3.5.0. This should not really affect the performance however and is no problem now, see: https://github.com/huggingface/transformers/pull/8518<|||||>Thank you for the clarification!