My approach above is suppose to be correct. I do check (write log) the learing rate each epoch to make sure it is schedule as expect. (use mlflow or comet.ml for fast log with chart, or just write to file)
For my use-case, actually learning rate scheduler does not help much (something fishy here). Result as same as no scheduler (fixed lr).
So, I don’t use scheduler anymore.
@Sefa_Alp, please make experiment and compare.