Could you, please explain then how to save and reload an optimizer.
I understand that if one is saving the model, the optimizer must also be saved otherwise the learning_rate value inside the optimizer is lost.
Thanks.