Multilingual T5 is not adding new tokens
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.11.2
- Platform:google colab
- Python version:3.7.12
- PyTorch version (GPU?):
- Tensorflow version (GPU?): 2.6.0
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?:No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 I am using mT5 model. The problem arised when I am trying to resize the token embeddings. I added new special tokens to my tokenizer's vocabulary, when I resized the model's token embeddings according to the length of the tokenizer, I am facing the issue.
Link: https://colab.research.google.com/drive/1ooKa5aQ_FAEnicxBL8bJnNAO4H9vFuv-?usp=sharing

Code from the notebook:
!pip install transformers
!pip install sentencepiece
from transformers import TFMT5ForConditionalGeneration, T5Tokenizer
model = TFMT5ForConditionalGeneration.from_pretrained("google/mt5-base")
tokenizer = T5Tokenizer.from_pretrained("google/mt5-base")
tokenizer.add_special_tokens({'bos_token':'','eos_token':''})
model._resize_token_embeddings(len(tokenizer))
The error message is:
ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.


 -->

## Information


The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.You can directly run the code & check the error coming from the last line.
2.Also you can go the colab notebook & have a look at the error as well.


related #13840<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.