You are already using a batch size of 1 in your DataLoader, so that you can’t lower it further.
If your model uses nn.BatchNorm layers, I would set the batch size to at least 16.
Are you sure your error CUDNN_STATUS_EXECUTION_FAILED is caused by running out of memory?
Could you check the memory of your cards with nvidia-smi while the code is running?
It would be strange, since you are only using a batch size of only 1 and the code was working before.
Also, you can add with torch.no_grad(): before the test sample evaluation so avoid storing the intermediate tensors.