nvidia-smi shows me using 8GB out of the 12 GB…
What puzzles me is that literally just stopped with a large minibatch size that training just fine, but then when I re-start the python script it craps out.
I am not using multiprocess data loading to my knowledge, I am using the pyTorch torch.utils.data.DataLoader basically, where I have set number of workers to 2.