I would assume your validation script wouldn’t have an optimizer so I’m unsure where you would call zero_grad() and step(). Also, if you are calling these operations right after another without a backward operation, the step() would only change the parameters if running stats were used so I’m also unsure why these operations should be executed.