One more thing regarding momentum. So if I’m correctly understanding that, the running mean and stddev are updated with each batch, and each new batch will contribute 10% (if momentum is 0.1) to the current mean and stddev. If that is the case then after all batches have gone through the network then the last batch had 10% of “contribution” to the final mean/stddev whereas the first batch that went through the network has been “washed out”. Is my understanding correct?