@ptrblck when i put loss.register_hook(lambda grad: print(grad)) before loss.backward() it gives me tensor(1., device='cuda:0'), is it what it is supposed to show? what intermediate values it is computing the gradient wrt?
@zhl515 and @ptrblck
I have a follow up question:
print(net.conv11.weight.grad) 

let me to print the grad values for conv11.weights, if i want to set these weights value to zero i thought i can do this:
Temp = net.conv11.weight.grad =net.conv11.weight.grad.clone()
net.conv11.weight.grad = torch.zeros(Temp.size())

but it is throwing

RuntimeError: assigned grad has data of a different type

can you please let me know your suggestion on that?
thanks
update:
I noticed that the second question is solved when i do the following 
net.conv11.weight.grad = torch.zeros(Temp.size()).to(device)