Can't get gradients from TF TransformerXL model forward pass
## 🐛 Bug (Actually I'm not very sure if it's a bug or am I doing something wrong)

<!-- Important information -->

- Model I am using : Transformer-XL

- Language I am using the model on (English, Chinese....): Chinese

- The problem arise when using: my own modified scripts

- The tasks I am working on is: my own task or dataset

## To Reproduce

0. My current testing environment is a CPU machine and a Cloud GPU machine I rented.
1. Clone my repository(https://github.com/Morizeyao/Decoders-Chinese-TF2.0) and install requirements.
2. Copy contents from scripts folder to root folder.
3. Run prepare_data.sh
4. Run train_xl.sh
5. You will see the print result from line 107 of file train_transformer_xl.py shows that the gradients are all zero.
6. And the loss don't change during the training process.

## Expected behavior

I have tested TFGPT2 model from Transformers library, it worked fine (just run train_gpt2.py). Only TFTransfomerXL has this problem.

## Environment

* OS: macOS and Ubuntu
* Python version: 3.7
* PyTorch version: NA
* PyTorch Transformers version (or branch): 2.2.1
* Using GPU ? No
* Distributed of parallel setup ? No

## And...

- Thank you guys for this awesome project!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
