Sorry for not being clear enough.
I meant you could wrap your code sample in a with torch.no_grad(); block and just call it after each iteration in your training loop. This would make sure the weights have the desired values in the next iteration.