LoRA training adapter_model.bin is 888 bytes always
### System Info

Ubuntu 22.04
Python 3.10.11
transformers 4.30.2
peft 0.4.0.dev0
accelerate 0.20.3

### Who can help?

@ArthurZucker 
@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Code is at https://github.com/kallewoof/alpaca-lora/blob/202306-ooba-imports/finetune.py

You need a text file with text content (i.e. not instruction-based). Anything goes.

Run the above script: `python ./finetune.py --base_model=MODEL --raw_data_path=TEXTFILE --batch_size=32 --micro_batch_size=1 --num_epochs=2 --lora_r=128 --lora_alpha=256 --cutoff_len=512 --overlap_len=256 --save_steps=1`


### Expected behavior

Expected: After one (each) iteration, a checkpoint should be saved with an adapter_model.bin file that contains the LoRA weights.

Got: The checkpoints are made, but the adapter_model.bin file is only 888 bytes and does not grow (there are two versions; both are wrong, one contains only the LoRA stuff, the other one contains optimizer.pt, rng_state.pth, scheduler.pt, etc. which I have no idea why they are saved for a LoRA.)

Note: using the exact same conda environment, Oobabooga is able to generate LoRAs after loading a model in 4 bit with double quant, where the above finetune.py fails to do so. I have also verified that the resulting raw data dataset is identical between the two code bases.
cc @younesbelkada <|||||>I backtracked until I was on the original tloen/alpaca-lora script, using the original dataset on int8 model, and I am still seeing the 888 byte adapter_model.bin file, along with a bunch of other files that you normally don't see in a LoRA output (optimizer.pt, etc).<|||||>I am closing this as I don't think I will be able to provide adequate feedback for a clean fix, and I've moved on to a different approach. Sorry for wasted time.