I thought I understood the answer at the time, but now I’m less sure.
Over which periods are the batch norm running statistics collected?
Are they reset or discarded each time I switch the model between modes?
This would work fine if you run through all your training images in one sequence (which is common, I guess), but I implement a sampling procedure of randomly drawing images from randomly chosen classes, in order to imitate a balanced training set. During training, I (frequently) switch back and forth between traning and eval mode, to monitor the progress.
I guess this is  a bad idea, if it resets running statistics all the time, and these really should be calibrated from the entire data set?