shubhvachher:

twork = torch.nn.utils.convert_sync_batchnorm(networ


Thanks a lot, since this new layer in pytorch1.1 operation is quite slow when apply to multi-node distributed training, so i do not plan to do so. speed can slow down to 2 times longer when i training on 4 node with each node 4 gpus. multi-node gpu communication is an bottleneck in my case.