Thank for your answer! It’s not clear to me what you mean with
“If you do that, you’ll also want to make sure you load all of your variables onto the same GPU to start with”
Which GPU are you talking about? I have, say, 3 of them, that I want to use. I have the input to the network; should I just call
input = input.cuda() 
model = DatsParallel(model) #I want to use all available GPUs anyhow
output = model(input)

or do something else?