Uncoupling ZeRO-3 weak ref bridge b/w Trainer and modeling_utils
https://github.com/huggingface/transformers/blob/fd6204b2a70d100800cb259a7fbddfc812631ed3/src/transformers/modeling_utils.py#L1168

So I briefly looked through the ZeRO-3 integration into the `Trainer` and this approach of creating a "bridge" between the `Trainer` and `modeling_utils` via a weak ref is neat.

However, what if I wanted to use ZeRO-3 with HF models outside the scope of the `Trainer` and `TrainingArguments`? Seems like I cannot at the moment.

There seems to be 4 places in `modeling_utils` where `is_deepspeed_zero3_enabled` is called, of which the only one that seems to be heavily tied to the custom `DeepSpeedConfigHF` class is the one referenced above.

Is it not possible to allow `Init()` args to come from the parent `from_pretrained()` method's `kwargs`?

@stas00 @sgugger 
Good call, @g-karthik!

Please see the decoupled PR here: https://github.com/huggingface/transformers/pull/11966

Now you can just do:
```
from transformers.integrations import HfDeepSpeedConfig
dsc = HfDeepSpeedConfigHF(ds_config)
model = AutoModel.from_pretrained(name)
```
and it'll just work, w/o needing the Trainer.

Please let me know if it works for you. 

> Is it not possible to allow Init() args to come from the parent from_pretrained() method's kwargs?

I can't see how this would work, since there several other core functions which rely on `is_deepspeed_zero3_enabled` and there is no way to pass this argument to those. That's why the "environmental" approach, rather than passing args.

The only other way I can see this solved is by tapping into the `model.config` object, but then it'll require a ton of code changes - e.g. all examples.<|||||>PR merged - I updated the comment above to reflect the final new name.

I will now work on updating the docs so it's all clear.