RWKV bug for 8-bit model fine-tuning.
### System Info

- `transformers` version: 4.30.0.dev0
- Platform: Linux-5.15.107+-x86_64-with-glibc2.31
- Python version: 3.10.11
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.9 (gpu)
- Jax version: 0.4.10
- JaxLib version: 0.4.10
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker  @younesbelkada @sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The 8-bit model inference works successfully, but after fine-tuning, the model fails when inferring it again.

Reproduction:
```
import torch
from transformers import AutoTokenizer, RwkvForCausalLM, GenerationConfig
from torch.optim import AdamW

model = RwkvForCausalLM.from_pretrained("RWKV/rwkv-raven-1b5", device_map="auto", torch_dtype=torch.bfloat16, low_cpu_mem_usage=True, load_in_8bit=True) 
tokenizer = AutoTokenizer.from_pretrained("RWKV/rwkv-raven-1b5")

optim = AdamW(model.parameters(), lr=1e-4)

ctx = "Hello my name is Bob"
inputs = tokenizer(ctx, return_tensors="pt").to(0)
model(inputs["input_ids"])   # ok

model.train()
outputs = model(inputs["input_ids"], labels=inputs["input_ids"])
loss = outputs.loss
loss.backward()
optim.step()

model.eval()
model(inputs["input_ids"])   # failed
```
or see my colab code as follows:
https://colab.research.google.com/drive/1l_vNHPd9_Z40dPkhIj5CxgrLhIn1Edyc?usp=sharing

### Expected behavior

After fine-tuning, the model should still work properly.
Hi @LetianLee 
Thanks for the issue! 
In fact, you cannot train a model that has been purely loaded in 8bit. In order to apply fine tuning using 8bit / 4bit models, you need to add adapters on top of the model and train these adapters only. 
Please check out the official example of PEFT: https://github.com/huggingface/peft/blob/main/examples/int8_training/Finetune_opt_bnb_peft.ipynb and adapt it to your needs. You may need to manually specify `target_modules=["key", "value", "receptance"]` when defining the `LoraConfig`. Please let us know how it goes<|||||>Hi @younesbelkada ,
Thank you for your kind reply and explanation. Since this is the case, I will close this ticket as it is not an issue related to Hugging Face/Transformers. Thank you very much for providing the relevant tutorial. I will now proceed to try the Lora fine-tuning. Thanks!<|||||>Thanks so much @LetianLee !