Extending the learning rate thing,  check the learning rate schedule, print out lr and calculate the value it is supposed to be by hand. Wrong training steps calculation causing wrong lr schedule takes me a whole month to debug…