Great, thank you, it worked out! which GPU device I use for backprop does not affect the results/backprop itself, right? Especially, because the whole composed model is: cnn1(cnn0(x)), i.e. the outermost cnn1 is in device cuda:1 and the innermost cnn0 is in device cuda:0. So, if I want a proper backward flow, should I put all the calculated losses in the device associated to the outermost cnn (cuda:1)? The following worked as well:
loss = loss_fn(outputs1, labels1).to('cuda:0') + loss_fn(outputs0, labels0)