Hello,
I am currently working on a project for computer vision to implement papers. I want to generalize the training script so that we can use multiple schedulers if we want.
I had 2 questions -

Is it absolutely necessary to save schedulers?
How to save/load multiple schedulers?

The workflow goes something like this -
train.yml
schedulers:
  # Scheduler1
  - name: "reduce_lr"
    mode: "min"
    factor: 0.1
    patience: 5
    verbose: True
  # Scheduler2
  # ..
  # Scheduler n 

Loading scheduler(s) initially
def get_scheduler(optimizer, scheduler_params):
  scheduler_params["name"] in SCHEDULER_DICT:
  scheduler = scheduler_params["name"]
  del scheduler_params["name"]
  logging.info("Using '{}' scheduler for this run.".format(scheduler.upper()))

  scheduler = SCHEDULER_DICT[scheduler](optimizer=optimizer, **scheduler_params)
  return scheduler

sch_dict = {}  # scheduler(s)
    for scheduler_params in config.schedulers:
        sch_name = scheduler_params["name"]
        sch_dict[sch_name] = get_scheduler(optimizer=optimizer, scheduler_params=scheduler_params)

Calling schedulers
for name, scheduler in sch_dict.items():
                if name == "reduce_lr":
                    scheduler.step(val_metrics["loss"])
                else:
                    scheduler.step()