Thanks to @ptrblck for your great solution answer.
I just want to remind that it also requires you to move your predicted results tensor and the ground-truth target tensor into the same GPU because of the loss function.
When the loss function calculates the difference between target and prediction tensor, it needs to have both of them on the same GPU.