I don’t know how you are adding SyncBatchNorm to the model, but in case you are using convert_sync_batchnorm, remove the operation and keep the standard batchnorm layers.
If you are manually adding SyncBatchNorm layers to your model, you might need to replace them with their non-sync equivalent and copy their internal states over.