Hey thanks a lot for this, I found this very helpful. Though is it possible to to apply a LR scheduler on one parameter group but not the other (meaning the other parameter group has a constant LR)?
I couldn’t figure it out, since the scheduler gets applied on all the parameter groups defined in your optimizer; so do you have any idea?