Training GPT or GPT-2 from scratch
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

I am trying to retrain GPT or GPT-2 from scratch, is there any implementation for this?
I think, just create an instance of the model (without loading from pretrained one), switch it to train mode and run. That's all.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I am currently trying to implement this as well. Once I figured it out i'll let you know !