Here’s some update on my debugging process.
What I found is that only fake_img sent to device cuda:0, but NOT alpha and real_img. I don’t understand why because I indeed send alpha.to(device) and real_img.to(device).
For fake_img, no matter I did fake_img.to(device) or not, it will always be on cuda:0. Even when I write fake_img.to(device=‘cpu’), its output still shows that fake_img on cuda:0. I think the reason for that because I fake_img is an output from netG, which has been sent to multiple GPUs by nn.DataParallel(). When I test outputs from netD, they are also on cuda:0 device.
I tried to print the size of real_img, fake_img, and alpha, as well as their values.
real_img size torch.Size([16, 3, 128, 128])
fake_img size torch.Size([16, 3, 128, 128])
alpha size torch.Size([16, 3, 128, 128])

real_img tensor([[[[0.8157, 0.8196, 0.8157,  ..., 0.8980, 0.9020, 0.9059],...[0.5608, 0.5647, 0.5608,  ..., 0.8314, 0.8275, 0.8196]]]])
fake_img tensor([[[[0.1948, 0.3402, 0.4361,  ..., 0.3754, 0.9775, 0.9215],...[0.5765, 0.6778, 0.4765,  ..., 0.0668, 0.1717, 0.1910]]]], device='cuda:0', grad_fn=<GatherBackward>)
alpha tensor([[[[0.2796, 0.2796, 0.2796,  ..., 0.2796, 0.2796, 0.2796],...[0.3608, 0.3608, 0.3608,  ..., 0.3608, 0.3608, 0.3608]]]])

Later I try to code on python prompt, instead of running a python script. The code is exactly copy & paste from the script that throws me error, however this time the real_img & alpha are on cuda:0. There must be an erro on my script that makes the real_img & alpha on cpu even though I send them to device. Below is the code I typed on python prompt:
>>> device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
>>> train_set = DatasetFromFolder('/dockerx/rtgan_data/folder5_all', if_resize=True, resize_val=256)
>>> train_loader = DataLoader(dataset=train_set, num_workers=4, batch_size=16, shuffle=True)
>>> for index, (data, target) in enumerate(train_loader):
...     real_img = Variable(target)
...     real_img.to(device)
...     print(real_img)

tensor([[[[0.8157, 0.8196, 0.8235,  ..., 0.9020, 0.9020, 0.9059],......[0.5686, 0.5686, 0.5725,  ..., 0.8235, 0.8235, 0.8235]]]], device='cuda:0')

So in this promot, the real_img is successfully sent to cuda:0. I don’t understand why in my sciprt, real_img, or alpha is not sent to cuda:0.
Thank you for helping!