hi @Gurkirt,
I am also stuck in a somewhat similar issue. Can you please suggest me if you have solved yours?
Here is my issue:

  
    
    
    Use global statistics of BatchNorm in training mode 
  
  
    I am working on a edge detection code and my batch_size=1 (due to GPU RAM’s limitation). I am using a pretrained ResNet as my backbone architecture. While training I want to use the global statistics of batch norm layer rather than batch statistics (as it won’t make much sense because of batch_size=1). 
One option is to set model.eval() in my training part so the network will use the global statistics of the batch norm layer. Given my network doesn’t have any dropout layers, is it the right thin…