Note that the learning rate curve that you post seems to be not the actual lr that is used during training, but a utility function that tests out different lr’s to see how that influences the loss function. Its goal is to help you find a good starting lr. In other words, the graph is not the learning rate changing over time, but the loss over different a different lr parameter. In that sense, it is very normal that that curve is different between checkpoints, architectures, and even datasets.