Hi,
Do I need to convert each batchnorm layer to its sync-bn version or Do I only need to convert the model?
I mean:
class Model(nn.Module):
def __init__(self):
     self.conv = nn.Conv2d(...)
     self.bn = nn.SyncBatchNorm.convert_sync_batchnorm(nn.BatchNorm2d(...))
...
model = Model()

Or this:
class Model(nn.Module):
def __init__(self):
     self.conv = nn.Conv2d(...)
     self.bn = nn.BatchNorm2d(...)
...
     model = nn.SyncBatchNorm.convert_sync_batchnorm(Model())

Which one is the correct usage ? Could I simply call the convert function on the whole model ?