danh:

When we call .cuda() all the parameters and buffers of the module are moved to the GPU:


self.expected_moved_cuda_tensor is neither a parameter nor a buffer, that’s why it’s device is unchanged. If you want to create a parameter and use it then you can do it as follows-
import torch
import torch.nn as nn

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.linear1 = nn.Linear(2, 1)
        self.linear1.weight = torch.nn.Parameter(torch.ones(2, 1))
        self.linear1.bias = torch.nn.Parameter(torch.zeros(1))

    def forward(self, x):
        x = self.linear1(x)
        return x

You can even use those parameters in forward method like-
import torch
import torch.nn as nn

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.weight = torch.nn.Parameter(torch.ones(2, 1))
        self.bias = torch.nn.Parameter(torch.zeros(1))

    def forward(self, x):
        # linear regression completely from scratch,
        # using parameters created in __init__
        x = torch.mm(x, self.weight) + self.bias
        return x

And moving above model to .cuda() does move model parameters to cuda-
model = Model()
model.cuda()
print(model.weight.device) # prints device(type='cuda', index=0)
print(model.bias.device) # prints device(type='cuda', index=0)