I just feel a bit confused about whitening, like PCA or similar techniques.
Common approach is to substract mean and divide std on dataset(i.e. mean is dataset mean, std is dataset std)
However, I feel we should do this on batch basis. because we use minibatch to feed into NN and each batch we cannot garentee the batch_mean and batch_std is the same as dataset_mean and dataset_std.
but i did not see anyone doing this on batch basis…
is there something wrong with my understanding? or i miss something?