Strange start token in MT5 generation
## Environment info

- `transformers` version: 4.2.2
- Platform: Linux
- Python version: 3.6
- PyTorch version (GPU?):1.7.1

### Who can help
 Text Generation: @patrickvonplaten @TevenLeScao
 T5: @patrickvonplaten
 
## Information

Model I am using (Bert, XLNet ...): MT5

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import MT5ForConditionalGeneration, MT5Tokenizer

model = MT5ForConditionalGeneration.from_pretrained("google/mt5-small")
tokenizer = MT5Tokenizer.from_pretrained("google/mt5-small")

text = 'summarize: Bidirectional Encoder Representations from Transformers is a Transformer-based machine learning technique for natural language processing pre-training developed by Google'
inputs = tokenizer([text], max_length=512, truncation=True, return_tensors='pt')
summary_ids = model.generate(inputs['input_ids'])
print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in summary_ids])
```
The output I got is ['<extra_id_0>.']
## Expected behavior
I tried a few input texts. The generated output always start with <extra_id_0>, which doesn't happen in t5 generation. Anyone knows how to solve it?

One more thing: this behavior still persists after I fine tuned the model on my own dataset<|||||>Hi @tomdzh 

first of all, unlike the original T5, mT5 is not pre-trained on any supervised downstream task (like summarization, translation etc), so generation work without fine-tuning it.

Also, it would be hard to answer why it's happening in fine-tuned model without looking at any code.<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.