Update, only the Address only in use error seem to remain if the last line is replaced with
srun -l --mpi=none --ntasks=2 --ntasks-per-node=2 singularity exec $SLURM_SUBMIT_DIR/PyTorch-1.8.1.sif python -m torch.distributed.launch --use_env tmp.py