You can reduce the memory usage during validation by wrapping the validation loop in a with torch.no_grad() block, which will make sure to not store the intermediate activations, which would be needed to calculate the gradients. If you aren’t using it already, you might be able to increase the batch size during validation further and speed up this loop.