Your code looks generally alright besides this line of code:
out2.to(gpu1)

While nn.Modules are transferred inplace, you have have to assign tensors back:
out2 = out2.to(gpu1)

Could you fix this line and see it it’s working?