Just found a quick hack:
suppose loss1 is the cpu tensor Variable, then we can directly set:
loss1.data = loss1.data.cuda()

The gradients are also correct. Verified by a simple toy example:
import torch
from torch.autograd import Variable

x1 = Variable(torch.rand(10), requires_grad=True)
x2 = Variable(torch.rand(10).cuda(), requires_grad=True)

x1.data = x1.data.cuda()

y = x1 + 2 * x2
y.backward(y.data.clone().fill_(1))

print(x1.grad)
print(x2.grad)