Thank You.
I just added the suggested modification. Now, I am getting -
RuntimeError: NCCL error in: ../torch/lib/c10d/ProcessGroupNCCL.cpp:514, invalid usage, NCCL version 2.6.3 
Full traceback -

Traceback (most recent call last):
  File "Distributed_code1.py", line 217, in <module>
    main()
  File "Distributed_code1.py", line 214, in main
    mp.spawn(train,args = (world_size,),nprocs = world_size,join=True)
  File "/opt/conda/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 200, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/opt/conda/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
    while not context.join():
  File "/opt/conda/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 119, in join
    raise Exception(msg)
Exception: 

-- Process 1 terminated with the following error:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 20, in _wrap
    fn(i, *args)
  File "/workdir/script/Distributed_code1.py", line 110, in train
    ddp_model_transfer = DDP(model_transfer, device_ids = [device])
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 303, in __init__
    self.broadcast_bucket_size)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 485, in _distributed_broadcast_coalesced
    dist._broadcast_coalesced(self.process_group, tensors, buffer_size)
RuntimeError: NCCL error in: ../torch/lib/c10d/ProcessGroupNCCL.cpp:514, invalid usage, NCCL version 2.6.3

I also did a change in how the model is loading in the following way -
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model_transfer = model_transfer.to(device)
ddp_model_transfer = DDP(model_transfer, device_ids = [device])