The zero_optimization section of the configuration file is the most important part (docs), since that is where you define
which ZeRO stages you want to enable and how to configure them. You will find the explanation for each parameter in the
DeepSpeed docs.
This section has to be configured exclusively via DeepSpeed configuration - the [Trainer] provides
no equivalent command line arguments.
Note: currently DeepSpeed doesn't validate parameter names, so if you misspell any, it'll use the default setting for
the parameter that got misspelled. You can watch the DeepSpeed engine start up log messages to see what values it is
going to use.
ZeRO-2 Config
The following is an example configuration for ZeRO stage 2:
json
{
    "zero_optimization": {
        "stage": 2,
        "offload_optimizer": {
