Hi,
I’m trying to use an optimizer only on a part of the module parameters. Should I call module.zero_grad() instead of optimizer.zero_grad() if there are other layers between the loss and layer that i’m training?
And what if I need to train only the last layer(right before the loss) of a module? The gradients for the previous layers wouldn’t be computed at all. So there is no difference between calling module.zero_grad() and calling optimizer.zero_grads(), right?