I have the same problem. How can we load the optimizer states when additional parameters are added to the models?
There is an attempt to do this (Add "strict" flag to ignore missing parameters in Optimizer.load_state_dict · Issue #34660 · pytorch/pytorch · GitHub 29) , however it its still not implemented, anyone has some updates on this?