Pretrained Transformer-XL gives unreasonable result on WikiText-103
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Transformer-XL (`transfo-xl-wt103`)

Language I am using the model on (English, Chinese ...): WikiText-103 (English)

The problem arises when using:
* [x] the official example scripts: `run_transfo_xl.py` (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official task: WikiText-103 (not GLUE/SQUaD)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Fix `run_transfo_xl.py` by removing the unused `work_dir` argument (line 52) and changing `lm_labels=target` to `labels=target` (line 108).
2. Run `run_transfo_xl.py`.
3. Observe the result: `test loss 10.20 | test ppl 26951.114`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

A reasonable result around the order of PPL=18.3, as reported in the paper. I know that the result will not be exactly the same, but something is definitely wrong here.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: Both 2.9.1 and Master (commit 384f0eb)
- Platform: Ubuntu
- Python version: 3.7.6
- PyTorch version (GPU?): 1.4.0 (CUDA 10.1, CuDNN 7.6.3)
- Tensorflow version (GPU?): -
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
