to the ZeRO technology, but instead uses tensor parallelism to scale models that can't fit onto a single GPU. This is a
work in progress and we will provide the integration once that product is complete.
Filing Issues
Here is how to file an issue so that we could quickly get to the bottom of the issue and help you to unblock your work.
In your report please always include:
the full Deepspeed config file in the report
either the command line arguments if you were using the [Trainer] or
   [TrainingArguments] arguments if you were scripting the Trainer setup yourself. Please do not
   dump the [TrainingArguments] as it has dozens of entries that are irrelevant.
Output of:
bash
python -c 'import torch; print(f"torch: {torch.__version__}")'
python -c 'import transformers; print(f"transformers: {transformers.__version__}")'
