Hi,
The statistics in the Batch normalization can be obtained in two ways:
1, using “track_running_stats”, then the average one over the training is used.
2, not using it, then the statistics of its individual batch is used.
I wonder how it can be done as in the original paper, “running over the dataset again and get the statistics”. In other words, how to “only update the statistics without updating other parameters” when running the model?
Thanks.