I try to change the size of a linear model by adding one new neuron after few epochs.
net = torch.nn.Linear(4, 4)
optimizer = torch.optim.SGD(net.parameters(), lr=0.01)
#training here ...
net.weight = torch.nn.Parameter(torch.cat((backSave.weight, torch.randn(1, 4)), 0))
net.bias = torch.nn.Parameter(torch.cat((backSave.bias, torch.randn(1)), 0))
#training here again but the weights do not change anymore

However, when I then train the model again after updating the size, the model weights no longer change. The model is not trained anymore.
I found a solution, I have to recreate the optimizer after the modification of the size
optimizer = torch.optim.SGD(net.parameters(), lr=0.01)

But I do not like this solution, is there another solution without defining again the optimizer
Thank you