just do like this:

Define Model
model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model)
model.cuda(args.gpu)
model_DDP = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu], find_unused_parameters=True)

save and load Model
torch.save(model_DDP, tmp.name)
model_DDP = torch.load(tmp.name)