@ptrblck Thanks for the pointer. My scenario is single node multi-gpu. Considering that case rank=0 and world_size=2. According to the imagenet example it says save the checkpoint if torch distributed is running or if torch distributed is running and the rank is equal to the num_gpus.



 ptrblck:

I’m not sure if the RANK env variable is useful at this point.


Why is not useful, my understanding is that it is set dynamically from the launch  utility and it will contain whichever rank is currently running?