Hi.Thanks for your answer. I can’t go with the second option because I want to freeze them before doing the backpropagation. However, for the first option, it dosen’t work as well. PyTorch still tells me that it can’t optimize these parameters. This is my code, and below is the error i get:
import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.layer1 = nn.Linear(3,5, bias = False)
    def forward(self, x):
        return self.layer1(x)
    
net = Net()
total_num = sum(p.numel() for p in net.parameters() if p.requires_grad)
print(total_num)                               # 15 parameters in total
print(list(net.parameters())[0][1:].numel())   # 12 parameters to optimize
param = list(list(net.parameters())[0][1:])
optim = torch.optim.SGD(param, lr = 0)

ValueError: can't optimize a non-leaf Tensor
Is there any other way?
Thank you in advance!