Sriharsha-hatwar:

I heard that we need to make sure the optimizer is also fed with this new learning rate if it needs to work


Could you link to the discussion, so that I can have a look?
In my example the learning rate of all param_groups will be reset, but I didn’t verify the code on the current master.