Can't find pretrained model for TFPegasusForConditionalGeneration 
## Environment info
- `transformers` version: 4.1.1
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.8.0
- PyTorch version (GPU?): 1.7.0+cu101 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
 Pegasus: @patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): pegasus-xsum

The problem arises when using:
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] my own task or dataset: (give details below)

## To reproduce
Try to download model for TFPegasusForConditionalGeneration 

Steps to reproduce the behavior:

1. Choose pegasus-xsum model
2. Fetch pretrained model
3.

```
import tensorflow as tf
from transformers import TFPegasusForConditionalGeneration, PegasusTokenizer
src_text = [
    """ PG&E stated it scheduled the blackouts in response to forecasts for high winds amid dry conditions. The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were scheduled to be affected by the shutoffs which were expected to last through at least midday tomorrow."""
]

model_name = 'google/pegasus-xsum'
tokenizer = PegasusTokenizer.from_pretrained(model_name)
model = TFPegasusForConditionalGeneration.from_pretrained(model_name)

batch = tokenizer.prepare_seq2seq_batch(src_text, truncation=True, padding='longest')
translated = model.generate(**batch)
tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)

```
## Expected behavior

The model should download.  Instead the model cannot be found
`404 Client Error: Not Found for url: https://huggingface.co/google/pegasus-xsum/resolve/main/tf_model.h5
`
Hey @demongolem, yes sadly those models were not yet uploaded in TF. Could you instead just run:

```python
import tensorflow as tf
from transformers import TFPegasusForConditionalGeneration, PegasusTokenizer
src_text = [
    """ PG&E stated it scheduled the blackouts in response to forecasts for high winds amid dry conditions. The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were scheduled to be affected by the shutoffs which were expected to last through at least midday tomorrow."""
]

model_name = 'google/pegasus-xsum'
tokenizer = PegasusTokenizer.from_pretrained(model_name)
model = TFPegasusForConditionalGeneration.from_pretrained(model_name, from_pt=True)

batch = tokenizer.prepare_seq2seq_batch(src_text, truncation=True, padding='longest', return_tensors="tf")
translated = model.generate(**batch)
tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)
```

Also note that you have to pass `return_tensors="tf"` in the tokenizer.<|||||>Thanks @patrickvonplaten .  The above code does do as necessary for me.  Thanks for pointing out the `return_tensors` part as well, I missed that one.