Have you tried saving the learning rate from the previous model and putting that in when you set up the optimizer instead of relying on the state dict?
optimizer = torch.optim.Adam(model.parameters(), lr=previously_saved_lr)