results for wikitext-2 clm using GPT-2 differ between paper and example code
Hi,

The example code [here](https://github.com/huggingface/transformers/tree/master/examples/language-modeling) to train causal LM on wikitext-2 uses a gpt2 base version (117M parameters). This code claims and also yields a perplexity of ~20 on the test data.

As you can see, the example script uses ```gpt``` as the ```model_name_or_path``` param which is actually the 117M model.
```python
python run_language_modeling.py \
    --output_dir=output \
    --model_type=gpt2 \
    --model_name_or_path=gpt2 \
    --do_train \
    --train_data_file=$TRAIN_FILE \
    --do_eval \
    --eval_data_file=$TEST_FILE
```

![image](https://user-images.githubusercontent.com/6441313/85228483-3c6d8980-b401-11ea-86f7-7f18e16b83a7.png)


However, the GPT-2 [paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf) [page 5, top table] shows that they got a perplexity of 29.41 with the said model.

![image](https://user-images.githubusercontent.com/6441313/85228388-d254e480-b400-11ea-87ac-29ab9064e794.png)

Did I miss something? Can you please explain me the mismatch?

Thanks in advance.

I think the results in this table of the GPT2 paper are zero-shot results (without fine-tuning). 