Solved by vmirly1 in post #2 
              
              
                Can you also try saving the scheduler state_dict, and then load that along with model and optimizer states? 
model = nn.Conv2d(3, 8, 3)
optimizer = optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
sched = lr_scheduler.MultiStepLR(optimizer, milestones=[10, 20, 30], gamma=0.1, last_epoch=-1
)
>>> …