legoh:

I see, so I have to register a hook on the tensor first? That will print the gradient with respect to the tensor right, not the parameters?


You have to register the hook before calling backward. And it will print the gradient of the tensor you called register_hook() on.

Also, I wonder if I could be facing zero gradients because of initializations in my weight matrix?

It is possible, if you’re in a local minima where the gradient in all direction is 0.