Solved by Seo in post #3 
              
              
                Finally I have solved. nn.DataParallel moves to the correct gpu only tensors, if you have list of tensors as input of your model forward() method, you need to move one by one tensors in the list on the correct gpu. The correct gpu can be retrieved by accessing the .device attribute of a tensor autom…