the learning rate is set to different values in different places. Command line rules. The values that get set are:
warmup_min_lr with the value of 0.
warmup_max_lr with the value of --learning_rate.
warmup_num_steps with the value of --warmup_steps if provided. Otherwise will use --warmup_ratio
  multiplied by the number of training steps and rounded up.
total_num_steps with either the value of --max_steps or if it is not provided, derived automatically at run
  time based on the environment and the size of the dataset and other command line arguments (needed for
  WarmupDecayLR).
You can, of course, take over any or all of the configuration values and set those yourself:
json
{
   "scheduler": {
         "type": "WarmupLR",
         "params": {
             "warmup_min_lr": 0,
             "warmup_max_lr": 0.001,
