My inital batch size was 8 but I tried it with 4 and 2 and had the same issue for both of them.
Yes before im running the code I clear the GPU memory with torch.cuda.empty_cache()
Is it maybe because Im running it on windows?