Solved by halahup in post #2 
              
              
                Ok, I have been able to solve the problem. It seems that the way my model class is written when I call get_activations() a new tensor is returned, hence the backward() doesn’t compute the gradient of the outputs with respect to the activations. I fix it by attaching a hook to the activation tensor w…