You can:
1/ Use the adaptive learning rate optimization method such as ADAM family
2/ A dynamic way to fine-tune learning rate while training (example: cycle learning rate: https://arxiv.org/abs/1506.01186)
3/ Switch the optimization while training with the best learning rate (https://arxiv.org/abs/1712.07628 1)