In fact, if I compute the relative difference between the parameters (weights, bias, running_mean, running_var) of the 3 last BN obtzained during the training phase (1st script) , and the debugging script (2nd script) I generaly got 1% to few per mille difference, except for 1 running_mean componant of the 3rd BN layers where I got in absolute value: tensor(-0.0068) versus tensor(0.0503) and tensor(0.0092) versus tensor(0.1384).
I do not know if these differences can explain what I see for the stats of outputs of the BN layers, and the losses. Anyhow, I will check the data pipeline as you suggest.
Thanks