No, these settings are not equivalent, as the first one will not freeze the trainable parameters (weight and bias), but will only use the internal running_mean and running_var.



 wwaayyaaww:

Unfortunately, the setting 2 doesn’t freeze the bn_layer from learning at all.
However, I observe all parameters and running statistics including running_mean, running_var, weight, and bias are remained unchanged.


I don’t understand this statement. If all parameters and the running stats remain the same, what do you consider is still “learning”?