a learning rate schedule changes the learning rate during learning and is most often changed between epochs/iterations. this is mainly done with two parameters: decay and momentum . there are many different learning rate schedules but the most common are time-based, step-based and exponential.