I have a large data set of 3D images that does not fit into memory. The channels of this dataset have very different scale and location. E.g. there is one channels with values in [0,1] and another one with values in [900,1200]. I would like to standardize each channel to have mean 0 and variance one.
I would prefer not to safe an extra copy of the dataset, but do the normalization on the fly. Something like batch normalzation, but where the mean/std over all previously seen examples is used instead of the current batch. How to do this?