BitsAndBytesConfig llm_int8_skip_modules does not work in the new version
### System Info

- `transformers` version: 4.30.0.dev0
- Platform: Linux-5.4.0-148-generic-x86_64-with-glibc2.17
- Python version: 3.8.16
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help?

@ArthurZucker @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import RobertaForSequenceClassification, BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(load_in_8bit=True, llm_int8_skip_modules=['classifier'])
model = RobertaForSequenceClassification.from_pretrained('roberta-large-mnli', quantization_config=quantization_config)
```

### Expected behavior

The 'classifier' layer should be in Float16 but is actually loaded in 8bit. This is problematic because it drastically lower the performance of the model. It also make it impossible to train it (using peft library for example).
Hi @AntoineBlanot 
Thanks for the issue and flagging! 
https://github.com/huggingface/transformers/pull/24043 should fix the issue!