ds_config = {...}  # deepspeed config object or path to the file
must run before instantiating the model
dschf = HfDeepSpeedConfig(ds_config)  # keep this object alive
config = AutoConfig.from_pretrained("gpt2")
model = AutoModel.from_config(config)
engine = deepspeed.initialize(model=model, config_params=ds_config, ...)
```
HfDeepSpeedConfig
[[autodoc]] deepspeed.HfDeepSpeedConfig
    - all
Main DeepSpeed Resources
Project's github
Usage docs
API docs
Blog posts
Papers:
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
ZeRO-Offload: Democratizing Billion-Scale Model Training
ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
Finally, please, remember that, HuggingFace [Trainer] only integrates DeepSpeed, therefore if you
have any problems or questions with regards to DeepSpeed usage, please, file an issue with DeepSpeed GitHub.
