Giulio:

I read that running_mean and running_var are buffers and do not require gradients. Is it true?


Yes, that’s true as the running stats will be updated in each forward pass if the module is set to training mode using the batch statistics.
By default batchnorm layers will contain trainable parameters (weight and bias), which will get gradients and will thus be updated. Setting their requires_grad attribute to False would freeze these parameters.