Hello all
I have a question about Fitting one large model (costing around 18G memeory) into multiple GPU, like 4 TITANX 12G;
Currently, I am not setting anything. The model itself contains one inception_v3 module, stacked convlstm and fc classfier. The pysudo code is like below:
Class Network(nn.Module):
    def __init__(self):
         self.enc = inception_v3().cuda()
         self.convlstm = ConvLSTM().cuda()
         self.discriminator = fully_connect_layer().cuda()

    def forward(self, input):
        x = self.enc(input)
        x = self.convlstm(x)
        x = self.discriminator(x)

        return x

The error I have is:
Expected tensor for argument #1 ‘input’ to have the same device as tensor for argument #2…;
It seems like pytoch would automatically allocate tensor that exceeds limit for gpu #1 to another gpus. However, recognize tensor from different gpu is an issue ? Please someone let me know how to deal with this.