Hi tueboesen,
The running_mean and running_var should be registered as buffers and not parameters. The update to these tensors happens inside the function call. And gradients are not available. Hence, the error.