Why can’t I just append my new parameter tensor list to the current optimizer’s state_dict?
model = MyModel()
optimizer = torch.optim.Adam(model.convblock0.parameters(), learning_rate)
state_dict = optimizer.state_dict()
for epoch in range(10):
    if epoch == 2:
        new_par = model.modules[0].parameters()
        state_dict['param_groups'][0]['params'].append(new_params)
...

Does this preserve previous state of the optimizer? Any help is appreciated. Thank you.
cc: @albanD @smth