Adding Transformer-XL pre-trained model
Add Transformer-XL (https://github.com/kimiyoung/transformer-xl) with the pre-trained WT103 model (maybe also the 1B-Word model).

The original Google/CMU PyTorch version  (https://github.com/kimiyoung/transformer-xl/tree/master/pytorch) has been slightly modified to better match the TF version which has the SOTA results. I've mostly untied the relative positioning/word biases and changed the initialization of memory states (TODO: PR these modifications back up to the original repo).

The Google/CMU model can be converted with:
```bash
pytorch_pretrained_bert convert_transfo_xl_checkpoint [PATH_TO_TRANSFO_XL_FOLDER]/model.ckpt-0 [PATH_TO_SAVE_PT_DUMP]
```
And the corpus and vocabulary with:
```bash
pytorch_pretrained_bert convert_transfo_xl_checkpoint [PATH_TO_TRANSFO_XL_DATA_FOLDER]/cache.pkl [PATH_TO_SAVE_DATA_AND_VOCABULARY]
```

The evaluation can be run with
```bash
cd ./examples
python eval_transfo_xl.py --cuda --model_name [PATH_TO_PT_DUMP] --work_dir [PATH_TO_SAVE_LOG]
```

Currently I have slightly higher values for the perplexity with the PyTorch model (using the TF pre-trained weights) `20.4` versus `18.3` for the TF version, might try to investigate this a little bit further.
Awesome work!

I'm not sure I understand how this model integrates with BERT. Did Google release weights for the MLM + next sentence prediction task using the Transformer XL? And if they did not, how well do the classical LM weights performs for finetuning tasks?<|||||>Oh it's not integrated with BERT, that just another model. I'm adding it here to make it use the same easy-to-use interface (with pretrained/cached model and tokenizer) like I do for the OpenAI GPT in the other PR (#183)
It's a language model like OpenAI GPT (trained with just a classical LM loss).
Check the Transformer-XL repo/paper for more details!<|||||>Ok, managed to find the bug in this conversion.
The custom-made AdaptiveSofmax used in Transformer-XL indexes the next-cluster probability tokens in reverse-order (see indexing by `-i` on [line 141 of the original repo](https://github.com/kimiyoung/transformer-xl/blob/master/pytorch/utils/proj_adaptive_softmax.py#L141)) so we got worse performances on less frequent words.
Tough to find!

Fixing this conversion we got a test set perplexity of `18.213` on Wiki-103 to be compared to a reported result of `18.3` with the TensorFlow model.<|||||>#254 is also the main PR for the inclusion of Transformer-XL. Closing this PR.<|||||>Hi, I am assuming the pre-trained model available with huggingface is the `large` variant and not `base`?