Solved by mrshenli in post #2 
              
              
                It depends on how the code was written. If the model forward function has sth like: 
def forward(input):
    x1 = self.layer1(input.to("cuda:0"))
    x2 = self.layer2(input.to("cuda:1"))
    x3 = self.layer3(input.to("cuda:2"))
    return x3

Then it would certainly fail as it cannot find the device…