What are the GPU RAM requirements of popular models?
# ❓ Questions & Help

What are the GPU RAM requirement of `gpt2`, `gpt2-medium`, `distilgpt2`, `bert-base-uncased` and/or `distilroberta-base`

* for training?
* for inference?

Additionally, how do you calculate or find this information for other models?

original StackOverflow question: https://stackoverflow.com/questions/61226569/what-are-the-gpu-ram-requirements-of-popular-huggingface-transformers-models
related: #1750
Hi @r0levrai,

Good question! We are actually thinking about a good visualization for exactly that. Maybe in 2,3 weeks :-) 

We already have a very useful script to test RAM requirements which you can find here: 
`https://github.com/huggingface/transformers/blob/master/examples/benchmarks.py`<|||||>It should work with any model for a given `batch_size` and `sequence_length`. Let me know if you encounter problems with the script!<|||||>any updates on that visualization? 👀 

Also that link 404's now.<|||||>Hey @thesofakillers - note that we don't support benchmarking utils in Transformers anymore