Thanks @ptrblck  for the prompt reply. I actually printed the size of the features in the model["decoder"] forward function but I found that batch_size was not reduced (implying the data not being distributed among multiple GPUs. To be specific features is a list of size 5,and each of its element is a 4 dimensional tensor(Batch Size x Channel x h x w). Point I am getting confused is, each element of the list (which is a tensor) has to shared across multiple gpu(in Dataparallel case), Which
I am not sure , how shall I go about this, since features.cuda() is an obvious invalid statement (  AttributeError: 'list' object has no attribute 'cuda')