Solved by ptrblck in post #4 
              
              
                self.input_gmm was also detached in the line before. 
The general rule is, as long as you use PyTorch functions, don’t detach the tensors (via recreating new tensors, calling .detach() or item()), Autograd will be able to track the computation graph and calculate the gradients.