Solved by Mazhar_Shaikh in post #2 
              
              
                Hi tueboesen, 
The running_mean and running_var should be registered as buffers and not parameters. The update to these tensors happens inside the function call. And gradients are not available. Hence, the error.