             "warmup_num_steps": 1000
         }
     }
}
But then you're on your own synchronizing the [Trainer] command line arguments and the DeepSpeed
configuration.
For example, for WarmupDecayLR, you can use the following entry:
json
{
   "scheduler": {
         "type": "WarmupDecayLR",
         "params": {
             "last_batch_iteration": -1,
             "total_num_steps": "auto",
             "warmup_min_lr": "auto",
             "warmup_max_lr": "auto",
             "warmup_num_steps": "auto"
         }
     }
}
and total_num_steps, warmup_max_lr, warmup_num_steps and total_num_steps will be set at loading time.
fp32 Precision
Deepspeed supports the full fp32 and the fp16 mixed precision.
Because of the much reduced memory needs and faster speed one gets with the fp16 mixed precision, the only time you
