to the following documentation.
You can find dozens of DeepSpeed configuration examples that address various practical needs in the DeepSpeedExamples
repo:
bash
git clone https://github.com/microsoft/DeepSpeedExamples
cd DeepSpeedExamples
find . -name '*json'
Continuing the code from above, let's say you're looking to configure the Lamb optimizer. So you can search through the
example .json files with:
bash
grep -i Lamb $(find . -name '*json')
Some more examples are to be found in the main repo as well.
When using DeepSpeed you always need to supply a DeepSpeed configuration file, yet some configuration parameters have
to be configured via the command line. You will find the nuances in the rest of this guide.
To get an idea of what DeepSpeed configuration file looks like, here is one that activates ZeRO stage 2 features,
