New update breaks T5, gpt2, opt models (probably all models actually) if bitsandbytes is installed
### System Info

It seems like a new update in the repo is causing an error for all models tested, including all the T5, gpt2 and opt models.

The error only occurs if bitsandbytes is installed, I tried an earlier version of bitesandbytes and same problem occured.

I made a colab to showcase it: https://colab.research.google.com/drive/1TSMLP3oPkAb-sBL_9l9KmXtRpP314Axc?usp=sharing

It must have been an update made in the past few hours, since code I used earlier today suddenly raised this error:

```
Traceback (most recent call last):
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/utils/import_utils.py", line 1030, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/models/t5/modeling_t5.py", line 36, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 88, in <module>
    from .utils.bitsandbytes import get_key_to_not_convert, replace_8bit_linear, set_module_8bit_tensor_to_device
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/utils/bitsandbytes.py", line 10, in <module>
    import bitsandbytes as bnb
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/bitsandbytes/__init__.py", line 6, in <module>
    from .autograd._functions import (
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 4, in <module>
    import bitsandbytes.functional as F
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/bitsandbytes/functional.py", line 14, in <module>
    from .cextension import COMPILED_WITH_CUDA, lib
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/bitsandbytes/cextension.py", line 41, in <module>
    lib = CUDALibrary_Singleton.get_instance().lib
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/bitsandbytes/cextension.py", line 37, in get_instance
    cls._instance.initialize()
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/bitsandbytes/cextension.py", line 15, in initialize
    binary_name = evaluate_cuda_setup()
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py", line 136, in evaluate_cuda_setup
    cc = get_compute_capability(cuda)
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py", line 112, in get_compute_capability
    return ccs[-1]
IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 462, in from_pretrained
    model_class = _get_model_class(config, cls._model_mapping)
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 359, in _get_model_class
    supported_models = model_mapping[type(config)]
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 583, in __getitem__
    return self._load_attr_from_module(model_type, model_name)
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 597, in _load_attr_from_module
    return getattribute_from_module(self._modules[module_name], attr)
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 553, in getattribute_from_module
    if hasattr(module, attr):
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/utils/import_utils.py", line 1020, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/transformers/utils/import_utils.py", line 1032, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.t5.modeling_t5 because of the following error (look up to see its traceback):
list index out of range
```

### Who can help?

@younesbelkada
@TimDettmers


### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

!pip install  git+https://github.com/huggingface/transformers.git
!pip install bitsandbytes==0.32.1

from transformers import AutoModel
model = AutoModel.from_pretrained("gpt2")

### Expected behavior

It should load the model, but it never does.
Hi @ViktorThink . Could you try if the following version works?

`bitsandbytes==0.31.8` or `bitsandbytes==0.31.5`<|||||>Oh, I got what was the issue now...

I had bitsandbytes installed on an instance with no cuda device, and that raised the error.

Thank you very much for the quick reply! Highly appreciated!<|||||>Thanks for pointing out this issue @ViktorThink !
This should be fixed in https://github.com/huggingface/transformers/pull/18859 that has been merged recently 💪 