Solved by ptrblck in post #4 
              
              
                It won’t follow these scheme, if you don’t call scheduler.step() in each epoch. 
Here is a small example: 
optimizer = optim.SGD([torch.randn(1, requires_grad=True)], lr=1e-3)
exp_lr_scheduler = optim.lr_scheduler.StepLR(optimizer,
                                             step_size=7, gamma=0.1)…