You don’t need to call torch.cuda.empty_cache(), as it will only slow down your code and will not avoid potential out of memory issues.
If PyTorch runs into an OOM, it will automatically clear the cache and retry the allocation for you.
That being said, you shouldn’t accumulate the batch_loss into total_loss directly, since batch_loss is still attached to the computation graph, which will be stored as well.
Use total_loss += batch_loss.detach() instead and rerun the code.
Also, you are right in wrapping the validation loop into a with torch.no_grad() block, as this will avoid storing the intermediate tensors, which would be needed for the backward pass.
If that doesn’t help, you might need to further decrease the batch size or use torch.utils.checkpoint to trade compute for memory.