If you don’t want to use SyncBatchNorm you could just remove the convert_sync_batchnorm call and keep the original nn.BatchNorm*D layers.
From the docs 14:

Helper function to convert all BatchNorm*D  layers in the model to torch.nn.SyncBatchNorm 14 layers.