   }
}
But then you're on your own synchronizing the [Trainer] command line arguments and the DeepSpeed
configuration.
If you want to use another optimizer which is not listed above, you will have to add to the top level configuration.
json
{
   "zero_allow_untested_optimizer": true
}
Similarly to AdamW, you can configure other officially supported optimizers. Just remember that may have different
config values. e.g. for Adam you will want weight_decay around 0.01.
Scheduler
DeepSpeed supports LRRangeTest, OneCycle, WarmupLR and WarmupDecayLR learning rate schedulers. The full
documentation is here.
Here is where the schedulers overlap between 🤗 Transformers and DeepSpeed:
WarmupLR via --lr_scheduler_type constant_with_warmup
WarmupDecayLR via --lr_scheduler_type linear. This is also the default value for --lr_scheduler_type,
