Hi chen! register_hook() is a function for Variable instance while register_backward_hook() is a function for nn.Module. If you want to save gradients, you can append them to a global list. You don’t need to remove it as long as you want to keep tracking the gradients.