Right.
I tried training the model using launch utility but I got no output. My training program (train.py) looks like:
def runTraining(i,args):
    torch.cuda.set_device(args.local_rank) 
    torch.distributed.init_process_group(backend='nccl',
                                     init_method='env://')   
....
net = nn.parallel.DistributedDataParallel(net)

and the script is:
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 ./src/train.py