example code for fine-tuning CLM does not work for GPT
## Environment info
- `transformers` version: 3.5.0
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.6
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: NO 

### Who can help
@patrickvonplaten @TevenLeScao


## Information

Model I am using: open-ai GPT. 

The problem arises when using:
* [x] the official example scripts: (give details below)

```
python run_clm.py --model_name_or_path openai-gpt --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 --do_train --do_eval --output_dir /tmp/test-clm
```

It's the example script on: [(https://github.com/huggingface/transformers/tree/master/examples/language-modeling], which is used to fine-tune a casual language model. 

## To reproduce

Steps to reproduce the behavior:

1. go to transformers/examples/language-modeling 
2. run the following command 

`python run_clm.py --model_name_or_path openai-gpt --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 --do_train --do_eval --output_dir /tmp/test-clm`

3 The following error occurs: 
`
RuntimeError: The size of tensor a (1024) must match the size of tensor b (512) at non-singleton dimension 1
`

In my case, the error does not occur when using `--model_name_or_path gpt2`. 

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

The progress bar should be filled and the language model should be finetuned. 

<!-- A clear and concise description of what you would expect to happen. -->

Could you try putting `--block_size=512` in your command to see if it changes something?<|||||>@LysandreJik  Thank you it solved the issue! 