Yes, at beginning, I call . cuda() too, but I ignore such a  problem. For example:
#let  module name:   Exam
x = Parameter(torch.FloatTensor(2,3).normal_())
y = torch.rand(2,3)
z = torch.mm(x, y.t())
this is, in module, we sometime defines some constants and compute it with the Paraeter.   Even call Exam.cuda(),  a runtime error will occur:
RuntimeError: Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #2 ‘other’
So we have to write:
y = torch.rand(2,3).cuda()
So how can I solve this kind of problem?