The scheduler might track multiple param_groups as described here 201.


It should still work as shown in this example:


model = nn.Linear(10, 2)
optimizer = optim.Adam(model.parameters(), lr=1e-3)
scheduler = optim.lr_scheduler.ReduceLROnPlateau(
    optimizer, patience=10, verbose=True)

for i in range(25):
    print('Epoch ', i)
    scheduler.step(1.)    
    print(optimizer.param_groups[0]['lr'])


No, you should still save the optimizer’s state_dict (and also call optimizer.step(), as the scheduler is not a replacement for the optimizer). Additionally, you could also store the scheduler’s state_dict.