Do not Preload a Deep Learning Framework
### Feature request

I need some toggle (say envvar `HF_FRAMEWORK` or configuration option `transformers.config.framework`) or runtime check on importing `transformers` in order not to import `tensorflow`, `torch`, and `flax` simulteneously.

### Motivation

At the moment `transformers ` package imports all deep learning packages even if they are not used. In other words, if there are `tensorflow` and `torch` installed simultaneously then both packages will be imported despite the only `torch` is actually used.

### Your contribution

I'll see but I am not sure that it is easy to remove explicit imports of DL framework on `transformer` imports.
Those environment variable exist, but Transformers also only imports the frameworks as needed, they are named `USE_TF`, `USE_TORCH` and `USE_JAX`.<|||||>Many thanks!

Are they described somewhere in documentation? I didn't manage to find them. Is it possible to add `HF_` prefix to be consistent with `HF_HOME` as an example?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.