I’ve got a model with 110M parameters, and I’m training it on a very small dataset (like 500 examples).
Yet that is enough to crash Pytorch on a K80 GPU with 11GB VRAM.
What is going on here? 110M x 4 (float size) = 440M = 0.440 GB + minuscule dataset size != 11GB VRAM…
Thanks for your help