jmaronas:

but as long as i do not now how this estimator approximate the true mean and true std,


I don’t know what exact measure of goodness you are looking for. Yes it is a smoothed moving average with momentum. If you want a theoretical guarantee, then unfortunately you can definitely construct examples where this approximation can be bad. That being said, it is usually proved to be sufficient in practice. So people don’t tend to worry about it.
They don’t even need to be close to true mean and var, as long as they give good results. Afterall, it’s just a smart “trick” for training networks. If you are worried about this, I think you should more worried about why BN work, what internal covariate shifts are, and what is causing these shifts.