These differences are most likely created by the limited floating point precision, not by batchnorm or any other layers.
Usually you would expect a relative error of ~1e-6, but if you are using a deep model, these errors might accumulate.