I guess my confusion is more of a fundamental understanding of gradients. Are the gradients obtained from [x.grad.data for x in network.parameters()] the same gradients used to update the parameters in gradient descent e.g. theta = theta - lr*gradient, where gradient is the derivative of the cost function w.r.t. the parameters in the network? Sounds like they’re something different - this point seems subtle to me so please correct any misunderstands I have - and that the gradients obtained from register_backwards_pass(hook) are exactly the gradients from theta = theta - lr*gradient, right?
Also say for whatever reason, if I want to manually implement dropout, which one should I use to control gradient updates, register_backwards_pass(hook) or [x.grad.data for x in network.parameters()]?
Like if I want to directly replace in_grad from register_backward_hook(hook), can I simply do something like:
def hook(module, in_grad, out_grad):
   in_grad = my_new_gradient

nn.Linear.register_backward_hook(hook)