Hey @ajayp1, I am not familiar with slurm, but RPC’s MASTER_ADDR and MASTER_PORT should be similar to the ones used by collective communications and DistributedDataParallel (DDP) in init_process_group 6. Have you previously used DDP with slurm?
Found an example here: Multi-node-training on slurm with PyTorch · GitHub 30