Hi everyone,
Please I would like to pass halve of the parameters in model.fc.weight of resnet model to Adam optimizer. To be specific, suppose the weights stored in model.fc.weight  is 1000 by 512, then I want to pass halve of the weights, which would be a 500 by 512 matrix, to Adam optimizer. I proceeded as follows:
w_0 = model.fc.weight[0: 501, :]
optimizer = torch.optim.Adam([w_0], lr = 0.001)
optimizer.step()
I get the following error:
ValueError: can’t optimize a non-leaf Tensor
Then I try:
w_0 = model.fc.weight[0: 501, :].retain_grad()
optimizer = torch.optim.Adam([w_0], lr = 0.001)
optimizer.step()
I get the following error:
TypeError: optimizer can only optimize Tensors, but one of the params is NoneType