It’s a known issue as described in the Warning in the docs 2:

The current implementation will not have the presented behavior for complex Module that perform many operations. In some failure cases, grad_input and grad_output will only contain the gradients for a subset of the inputs and outputs. For such Module, you should use torch.Tensor.register_hook() 2 directly on a specific input or output to get the required gradients.