I already find the reason.
pytorch1.2.0, ubuntu16,cuda10.
I use  multiprocessing  and DistributedDataParallel in my  code  rewrited  as https://github.com/pytorch/examples/blob/master/imagenet/main.py 1.
When I get the pth file, I print some  BN layers’ weight , I found it is exactly a same value.
That not happend when I use single gpu to train.
Someone can help me??