Thanks, I guess I don’t fully understand how batchnorm works during training/evaluation in pytorch. During training, batchnorm accumulates the history of means/stds and normalizes the inputs using the current. From the doc: `

‘by default, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation’

Does it mean than only one mean/std  (‘compound’) is used during the evaluation? I noticed Pytorch stores either 64 or 256 means/std.