Solved by alex.veuthey in post #2 
              
              
                That’s actually pretty straightforward, but I can understand not thinking about it, because it’s not explicit from the documentation. 
What you want to do is define your StepLR normally in the beginning, but instead of having values correspond to epochs, you set them up w.r.t. batches. Then you call…