Neta_Zmora:

In other words, this performs a similar function as optimizer.step(), using the gradients to updates the model parameters, but without the extra sophistication of a torch.optim.Optimizer. If you use the above code, then you should not use an optimizer (and vice-versa).


What’s the reason for not using the optimizer.step after clipping the gradients?