I think if you wanted to disable the gradients for validation you might want to do torch.no_grad() for one of the steps rather than just conditionally skipping the calls to losses.backwards() and optimizer.step.  Or, alternatively, set model.eval() before running the evaluation step and then set model.train() when you’re done.