run_clm.py crashes server.
I was running the run_clm.py script with deepspeed for finetuning gpt2-small on a 300GB data. I had provided 5 GPUs for the process, but as soon as the preprocessing of the data was completed the server crashed and I had to get it restarted. Can you point me to where we need to make changes to the script to support finetuning with such large datasets?
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.