t5 fast tokenizer save_vocabulary fails without sentencepiece file
## Environment info

- `transformers` version: 4.9.0.dev0
- Platform: Linux-5.4.0-1043-gcp-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version (GPU?): 1.9.0+cu102 (False)
- Tensorflow version (GPU?): 2.5.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.3.4 (tpu)
- Jax version: 0.2.16
- JaxLib version: 0.1.68
- Using GPU in script?: no (tpu)
- Using distributed or parallel set-up in script?: I guess data parallel

### Who can help

Models:
- t5: @patrickvonplaten

Library:
- tokenizers: @LysandreJik

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [] my own task or dataset: (give details below)

Task is summarization

## To reproduce

Steps to reproduce the behavior:

1. Use the [summarization example code](https://github.com/huggingface/transformers/blob/3cd15c1dd62c5c9a9202fae9f00b8eba3eb2b95d/examples/pytorch/summarization/run_summarization.py) and fine tune a pre-trained t5 tokenizer and model created according to the flax mlm example scripts and [t5 tokenizer](https://github.com/huggingface/transformers/blob/master/examples/flax/language-modeling/t5_tokenizer_model.py) -- for instance [t5-base-norwegian](https://huggingface.co/patrickvonplaten/t5-base-norwegian/tree/main)

When the finetuning-summary-trainer saves the model, it will also attempt to save the vocabulary. This will fail with the following stack trace, because the tokenizers `self.vocab_file` is None, where it is expected to point at a sentencepiece file:

```
Traceback (most recent call last):
  File "/home/yeb/Developer/yhavinga/t5-base-dutch-summarization/run_summarization.py", line 620, in <module>
    main()
  File "/home/yeb/Developer/yhavinga/t5-base-dutch-summarization/run_summarization.py", line 545, in main
    trainer.save_model()  # Saves the tokenizer too for easy upload
  File "/home/yeb/Developer/yhavinga/t5-base-dutch-summarization/transformers/src/transformers/trainer.py", line 1883, in save_model
    self._save(output_dir)
  File "/home/yeb/Developer/yhavinga/t5-base-dutch-summarization/transformers/src/transformers/trainer.py", line 1933, in _save
    self.tokenizer.save_pretrained(output_dir)
  File "/home/yeb/Developer/yhavinga/t5-base-dutch-summarization/transformers/src/transformers/tokenization_utils_base.py", line 1958, in save_pretrained
    save_files = self._save_pretrained(
  File "/home/yeb/Developer/yhavinga/t5-base-dutch-summarization/transformers/src/transformers/tokenization_utils_fast.py", line 567, in _save_pretrained
    vocab_files = self.save_vocabulary(save_directory, filename_prefix=filename_prefix)
  File "/home/yeb/Developer/yhavinga/t5-base-dutch-summarization/transformers/src/transformers/models/t5/tokenization_t5_fast.py", line 150, in save_vocabulary
    if os.path.abspath(self.vocab_file) != os.path.abspath(out_vocab_file):
  File "/usr/lib/python3.8/posixpath.py", line 374, in abspath
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Process finished with exit code 1
```

The following hack works around the problem:
```
diff --git a/src/transformers/models/t5/tokenization_t5_fast.py b/src/transformers/models/t5/tokenization_t5_fast.py
index 3f972b006..cc238a119 100644
--- a/src/transformers/models/t5/tokenization_t5_fast.py
+++ b/src/transformers/models/t5/tokenization_t5_fast.py
@@ -147,9 +147,10 @@ class T5TokenizerFast(PreTrainedTokenizerFast):
             save_directory, (filename_prefix + "-" if filename_prefix else "") + VOCAB_FILES_NAMES["vocab_file"]
         )
 
-        if os.path.abspath(self.vocab_file) != os.path.abspath(out_vocab_file):
-            copyfile(self.vocab_file, out_vocab_file)
-            logger.info(f"Copy vocab file to {out_vocab_file}")
+        if self.vocab_file:
+            if os.path.abspath(self.vocab_file) != os.path.abspath(out_vocab_file):
+                copyfile(self.vocab_file, out_vocab_file)
+                logger.info(f"Copy vocab file to {out_vocab_file}")
 
         return (out_vocab_file,)
 ```

## Expected behavior

No error.

Maybe of interest to @SaulLu :)<|||||>Thank you very much for sharing this error! I share your opinion.

 I have opened a PR  #12806 to solve this problem. :slightly_smiling_face: <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>unstale<|||||>Stumbled upon the same issue as I was using an old version of transformers -- Thanks a bunch for your PR @SaulLu ! Once I upgraded the transformers version, it got fixed. 