MT5-base tokenizer can't decode to target language after decoding
## Environment info

- `transformers` version: 4.9.0
- Platform: Google Colab
- Python version: 3.8+
- PyTorch version (GPU?):  1.9.0+cu102
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help


Models:

- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj


Library:

- tokenizers: @LysandreJik


## Information

Model I am using (MT5):

The problem arises when using:
* [ ] my own modified scripts: When I am finetuning mt5-small model for question answering using mt5ForConditionalGeneration, after running inference, the output is not in the specified language. 

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (Question Answering)

## To reproduce
link to my notebook: [link](https://colab.research.google.com/drive/12nMMdHul4Avxn38o3LZhsVgVE02I6g2E?usp=sharing)
Steps to reproduce the behavior:

1. Run the inference section
2. Run on any language
3. The model outputs in a mixed language

## Expected behavior
The expected behavior should be to produce output on a single language.
