       }
   }
}
Note that the command line arguments will set the values in the configuration file. This is so that there is one
definitive source of the values and to avoid hard to find errors when for example, the learning rate is set to
different values in different places. Command line rules. The values that get overridden are:
lr with the value of --learning_rate
betas with the value of --adam_beta1 --adam_beta2
eps with the value of --adam_epsilon
weight_decay with the value of --weight_decay
Therefore please remember to tune the shared hyperparameters on the command line.
You can also set the values explicitly:
json
{
   "optimizer": {
       "type": "AdamW",
       "params": {
         "lr": 0.001,
         "betas": [0.8, 0.999],
         "eps": 1e-8,
         "weight_decay": 3e-7
       }
