I used the resnet_fpn backbone from torchvision and I am trying to implement faster rcnn FPN by myself. I didn’t see any SyncBatchNorm layers in torchvision resnet. So I guess there is no communication.
I just validated 15 samples just to debug, it took less than 1 mins.
Yes, the log shows that the rank 0 finishes barrier successfully but rank 1 doesn’t.