Hi, I am using apex and multi-node multi-gpu training.
I wonder what’s the recommended way to setup sync_bn across nodes/cards.
In Nvidia’s official apex Imagenet example 2, it uses apex.parallel.convert_syncbn_model()
In torchvision’s official video-classificaiton example 9, it uses torch.nn.SyncBatchNorm.convert_sync_batchnorm
What’ the difference, and the recommended usage for distributed training?
Thanks.