OOM when running causal language modelling sample
The example at:

https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling#gpt-2gpt-and-causal-language-modeling

...mentions taking half an hour on a K80. I'm using an M40 24GB, which I would reasonably believe has sufficient VRAM, but it dies with OOM at startup. I used the exact commandline shown on the page.

Dropping -per_device_train_batch_size and --per_device_eval_batch_size from 8 to 4 succeeds, but it's still using ~15GB of VRAM

Unsure whether the WikiText-2 dataset has changed, or transformers has changed, or the model has changed, or there's something different about VRAM usage (eg default data type size) between the K80 and M40 24GB.

I'm just beginning with transformers fine-tuning which is why I was running the example commandline. Apologies if I'm missing something obvious.

Ubuntu 22.04 LTS
Python 3.10.6 running under venv
transformers 4.27.0.dev0-py3.10.egg
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.