I’m trying to train a model with batch normalization.
Though, one sample is quiet memory consuming and I cannot train with enough number of batch to apply batch normalization.
So, I’m thinking following steps.

Feed some samples and calculate running mean, var, and other parameters.
Copy above parameters to models, then, make batchnorm layers eval mode.

Here, my questions are

Even if batchnorm layers are eval mode, does autograd works correctly?
Is there any efficient way to achieve those process with pytorch or any other functions?

Any help would be appreciated.
Thanks.