Solved by smth in post #2 
              
              
                On 1: in training mode, BatchNorm DOES NOT use running_mean and running_std to compute the output. It only updates running_mean and running_std values internally. Hence, what you see is exactly what’s expected. 
On 2: At test time, the output is not the same as the last training output. The output u…