Hi @alex.veuthey and thanks for your answer.
I did already. Even though I had to repeat training from the beginning, but I wanted to give it a try. Comparing the results from the first epoch w and w/o these extra parameters, I find that it does not affect. However, if I save the optimizer using torch.save and then load it back again using torch.load, there is an error. Most probably because the optimizer has the parameters saved in its dictionary but isn’t updating them. Not sure about this.