I don’t think there is a general rule which says how to set the learning rate. As far as I understand, this is something that you learn using trial and error to build up your intuition about the dataset and the network you use.