Najeh_Nafti:

In validation without backward(), optimizer.step(), and optimizer.zero_grad() that I put in training?


Yes, during the validation you wouldn’t update the model, as this would be a data leak.
It’s still unclear in which setup you are hitting the OOM.
In any case, wrap the validation loop into with torch.no_grad() and check the memory usage via adding print(torch.cuda.memory_summary()) into the validation code to see how large the actual memory requirement is and which line of code causes the OOM error.