@ptrblck ,
Thank you!
I did the exmperments like this:

hook function to register_forward_hook,
if nn.BatchNormal2d, save data and stop saving after the BN.
The exmperiments is based on resnet18, and the upper picture is for the first bn of resnet18.

I tried another experiment:
during model loading, I copied the right running_mean and running_var to the left model, the output of bn of the left model is identical with the right.