tengerye:

If I just want to save a model, I don’t need dist.barrier() , right?


Yep, that should be fine. If only rank 0 saves the model and that might take very long, you can set the timeout argument in init_process_group 3 to a larger value. The default is 30min.