Hi there!
I have a question that might be simple to you: when trying to use a very large batch size (say 5000 for cifar10), what can I set in PyTorch in order to prevent the GPU from out of memory?
Like using a dtype that taking less memory or setting smaller num_of_worker?
Cheers!