Essentially the answer is yes. At train-time, mean and std will be estimated given the batch. Therefore a larger batch size will be more effective for this technique. Given enough training, you’ll probably have good estimates of your mean and std for your distribution.
However, I don’t see many circumstances where you couldn’t directly compute the mean and std. Even if the dataset is massive, you can compute mean/std precisely without loading everything up in memory. You can do one pass at the dataset to precisely compute the mean and perform another pass to compute the std.