ruotianluo:

And for batchnorm, train(True) uses batch mean and batch var; and train(False) use running mean and running var.


are the mean and std at batch level even the requires_grad is False?
For example, the batch norm layer N of a model has mean and std of 1.0 and 2.0 respectively. This layer is frozen (requires_grad is False). I feed input batch of mean and std of 0.3 and 0.5 to the network. Which values of mean and std will be used in Batch Norm Layer N?