Hi Sebastian Raschka,
Thanks for your answer. Yes it should work that way,but I’d like to save training time and not perform backprop on the weights i want to freeze. So the whole thing needs to happen before loss.backward(). If i do as you mentioned, or i set the gradients to zero before optimizer.step(), it still works, but there is no point because the backpropgation on all weights is happening anyways when calling loss.backward().
It seems that this is not possible in PyTorch by any means.
If there is anyone from the community that has a solution to this, it would be great if you share it here!