I have found that if we define a nn.Parameter on GPU in a Module, it will not appear in the list(parameters()). But if it is on cpu, it will. Why is the GPU version of nn.Parameter not implemented? How could we use nn.Parameter on GPU?
Test Code:
from torch import nn
from torch import Tensor
class M(nn.Module):
    def __init__(self):
        super(M, self).__init__()
        self.a = nn.Parameter(Tensor(1)).cuda()
m = M()
list(m.parameters())


class G(nn.Module):
    def __init__(self):
        super(G, self).__init__()
        self.a = nn.Parameter(Tensor(1))
g = G()
list(g.parameters())