pass a nested dict. This allows you to create the configuration on the fly and doesn't require you to write it to
the file system before passing it to [TrainingArguments].
To summarize you can do:
python
TrainingArguments(..., deepspeed="/path/to/ds_config.json")
or:
python
ds_config_dict = dict(scheduler=scheduler_params, optimizer=optimizer_params)
TrainingArguments(..., deepspeed=ds_config_dict)
Shared Configuration
This section is a must-read
Some configuration values are required by both the [Trainer] and DeepSpeed to function correctly,
therefore, to prevent conflicting definitions, which could lead to hard to detect errors, we chose to configure those
via the [Trainer] command line arguments.
Additionally, some configuration values are derived automatically based on the model's configuration, so instead of
