Solved by ptrblck in post #9 
              
              
                That’s expected, as the with torch.no_grad() block won’t switch any attributes of the model parameters, but will instead make sure that new operations performed inside this block won’t be tracked. Also, if you don’t call backward() on the output or any loss tensor and optimizer.step(), the paramete…