I’m manipulating it manually as I restart learning when validation loss stops decreasing. I’m just confused what is the right way to manually adjust learning rate, specially when using Adam. I did use ReduceLROnPlateau, but it didn’t result in desired behavior. I didn’t know that behavior about Adam. Thanks for clearing it up!