Hi @AndreaSottana
When using the distributed training mode, one of the processes should be treated as the main process, and you can save the model only for the main process.
Check one of the torchvision’s examples 3, which will give you a good idea for your problem.
Say you’re using two GPUs in the distributed training mode, and then there will be two processes of your script.
Both the processes will call save_on_master function in the above example at some time, but only one of the processes will save the model (i.e., when is_main_process() returns True)