Hi @yoshitomo-matsubara
I’d have a small follow up from this.
I’ve noticed that you don’t call dist.barrier() before save_on_master (for example here 1) but suggested I do that in my code. Is there a reason why you don’t need to do this in your code?
I’ve also noticed that when you save the model, you save the model_without_ddp.state_dict() instead of simply model.state_dict(). I’ve not seen this done before. Is there any advantage in doing this over simply saving the DDP model’s state dict?
Thanks again