If you see other usages of any SyncBatchNorm calls, I would remove them as well.
Yes, convert_sync_batchnorm converts the nn.BatchNorm*D layers to their sync-equivalent.
If you don’t want to use this, just keep the model as it is without calling any SyncBatchNorm functions and it will use the standard nn.BatchNorm*D layers.