When finetuning model from a pretrained one, the CAFFE users would set the BN layers with “use_global_stats: true”, which will employ the mean and std values in the pretrained model for the finetuning stage. In my works, I find that setting is sometimes important for performance. What should I do when using Pytorch if I want to employ the already-learned mean and std rather than the moving average in finetuning jobs?