Having problem Pre-training GPT models
## Environment info


- `transformers` version: 4.9.2
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.11
- PyTorch version (GPU?): 1.9.0+cu102 (True)
- Tensorflow version (GPU?): 2.6.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): EleutherAI/gpt-neo-2.7B 

The problem arises when using:
* [X ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X ] my own task or dataset: (give details below)

## To reproduce
I used a csv file with each line have a sample of example

Steps to reproduce the behavior:

My input: `!python /content/transformers/examples/pytorch/language-modeling/run_clm.py --model_name_or_path EleutherAI/gpt-neo-2.7B  --train_file /content/df.csv --output_dir /tmp/test-clm` 

i also tried using the no trainer version but still doesn't work.
What am i doing wrong?

What i got back:
```

Traceback (most recent call last):
  File "/content/transformers/examples/pytorch/language-modeling/run_clm.py", line 520, in <module>
    main()
  File "/content/transformers/examples/pytorch/language-modeling/run_clm.py", line 291, in main
    cache_dir=model_args.cache_dir,
  File "/usr/local/lib/python3.7/dist-packages/datasets/load.py", line 830, in load_dataset
    **config_kwargs,
  File "/usr/local/lib/python3.7/dist-packages/datasets/load.py", line 710, in load_dataset_builder
    **config_kwargs,
  File "/usr/local/lib/python3.7/dist-packages/datasets/builder.py", line 271, in __init__
    **config_kwargs,
  File "/usr/local/lib/python3.7/dist-packages/datasets/builder.py", line 370, in _create_builder_config
    builder_config = self.BUILDER_CONFIG_CLASS(**config_kwargs)
TypeError: __init__() got an unexpected keyword argument 'keep_linebreaks'
```
## Expected behavior

Just want to further train the GPT model

notebook: https://colab.research.google.com/drive/1bk8teH0Egu-gAmBC_zlvUifMHS7y_SyM?usp=sharing

Any help is much appreciated
Probably caused by this PR: #13150 

cc @stefan-it <|||||>I'm looking into it right now :)<|||||>Oh no, this is only happening when using CSV files as input :thinking: 

@mosh98 As a very quick workaround, could you try to "convert" your csv file into a normal text file (file extension .txt) and then re-run the training :thinking: <|||||>The `keep_linebreaks` argument is only implemented for text files in :hugs: Datasets:

https://github.com/huggingface/datasets/blob/67574a8d74796bc065a8b9b49ec02f7b1200c172/src/datasets/packaged_modules/text/text.py#L19

For CSV it is not available:

https://github.com/huggingface/datasets/blob/67574a8d74796bc065a8b9b49ec02f7b1200c172/src/datasets/packaged_modules/csv/csv.py<|||||>I'm working on a fix now (so that `keep_linebreaks` is only used when file extension is `.txt`)<|||||>Sure i can try that, i do have aquestion tho,

 when i convert my csv into a text file how will i organize it so that it uses each line as a sample  and also what command do i have to put when i run the script? 

At the moment i have each row is the csv file as an individual sample<|||||>You can use the same structure (one individual sample per line) for the text file. 

Command would be pretty much the same, but you need to use the file ending `.txt`, so that the training script will infer the correct extension for the `load_dataset` argument :)<|||||>Thank you @stefan-it  the script works now, running out of cuda memeory tho but i think it's irrelevant to the actual script and more to do with my device. 

Thanks Again!
