I don’t think DataParallel will give you any performance advantages, if your models work in a sequential way (input -> model1 -> model2 -> output).



 jmaronas:

For the moment what I do is put model1 in gpu 0 and model2 in gpu 1 and transfer the output from model1 to gpu1.


That’s what I would do. Do you get the error message using this approach?
If so, try:
model1 = model1.cuda(0)
model2 = model2.cuda(1)

for data in trainloader:
    data = data.cuda(0)
    data = Variable(data)
    output = model1(data)
    output = output.cuda(1) # transfer it to GPU1
    output = model2(output)