Hi
I have the following code:
h = model.classifier.weight.register_hook(lambda grad: grad * 0)
model.zero_grad()
loss.backward()
optimizer.step()

where model.classifier is a single fully-connected layer.
The purpose of this piece of code is to investigate how to use modified gradient to update parameters.
If I understand this correctly, after I set the gradient to 0, the weight will not be updated after each iteration. But, after I run this code, the weight still update normally, which means the optimizer still use the old gradient.
I also explicitly set the gradient to a new value, like:
model.classifier.weight._grad.data = torch.zeros(model.classifier.weight.size()).cuda()
and it doesn’t work either.
So, my question is how to pass the modified gradient to the optimizer? Thank you.