Write with transformers demo hardware
Hi,
Your package is great and your demo is cool too. I was wondering what kind of hardware does it take to generate text with the large language models. I've had a difficult time running GPT-2 with 1.5B params on a Tesla T4 GPU.
Any pointers will be greatly appreciated.
Thanks in advance,
Barak
Hi! We run on K80 and V100 GPUs. Running the 1.5B params model was very costly and hard to maintain, so we're not running it anymore, however. It should run fine on a single V100/Titan RTX however.

Did you have issues with the T4 GPU because of memory?<|||||>Thanks for you response. This is really beneficial. I think that it is due to memory. 
Are you serving your models with a flask server, TF-serving, or a different serving framework?
Were you serving it using your PyTorch or Tensorflow implementation?

Thanks again<|||||>We're serving our models in PyTorch, using a mix of gunicorn/falcon to handle requests. You can see the detailers [here](https://medium.com/huggingface/scaling-a-massive-state-of-the-art-deep-learning-model-in-production-8277c5652d5f)!<|||||>Really clear blog post.
Thanks