kazem:

As for your last two comments I did use just rank 0 to save the ddp, but I saved the state_dict() for ddp itself (without .module). That is why when I used your script I also had to remove the. Is it correctly to do so?


Yes, that is correct. The saved and loaded model type need to match.