My finetuned gpt2 model is taking wayy too long to generate samples, like 5-8 minutes
I fine tuned the gpt2 model using transformers, i trained it on a lyrics dataset, and after successful training, when i do model.generate(args), it takes like a hell lot of time to genrate results
What Should i do?

Hi @Krish-Nerkar , [this](https://discuss.huggingface.co/t/speeding-up-gpt2-generation/470) might help<|||||>@patil-suraj Thanks Alot! will check those methods out

