Error in finetuning starcoder with 8 GPU 24GB Memory 
### System Info

I am trying finetuning starcoder , with 8 GPU P40 (each 24GB Memory) , I am using
 "https://github.com/Xirider/finetune-gpt2xl " and also referring the "https://github.com/bigcode-project/starcoder"
 Facing error with both. 
 
 The error says :
 raise ValueError( ValueError: You can't train a model that has been loaded in 8-bit precision on multiple devices in any distributed mode. In order to use 8-bit models that have been loaded across multiple GPUs the solution is to use Naive Pipeline Parallelism. Therefore you should not specify that you are under any distributed regime in your accelerate config.

@younesbelkada , please help on the same 
Thanks!!!

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

/group-volume/orc_srib/mukesh.sm/STARCODER/finetune-gpt2xl/deep/lib/python3.8/site-packages/accelerate/accelerator.py"
python -m torch.distributed.launch \
  --nproc_per_node 8 finetune.py \
  --model_path="bigcode/starcoder"\
  --dataset_name="HuggingFaceH4/CodeAlpaca_20K"\
  --split="train"\
  --size_valid_set 10000\
  --streaming \
  --seq_length 2048\
  --max_steps 1000\
  --batch_size 4\
  --input_column_name="prompt"\
  --output_column_name="completion"\ 
  --gradient_accumulation_steps 16\
  --learning_rate 1e-4\
  --lr_scheduler_type="cosine"\
  --num_warmup_steps 100\
  --weight_decay 0.05\
  --output_dir="./checkpoints" \


### Expected behavior

I am not able to understand why its not getting finetuned either 8 bit or 4 bit .
Because full precision if i try gives out of memory error . 
Please help with the right steps to finetune the starcoder . 
Hi @22Mukesh22 
Thanks for the issue, Per my understanding you want to use NPP(Naive Pipeline Parallelism) 
For reference check: https://github.com/huggingface/accelerate/issues/1515#issuecomment-1584515731 and the entire thread
In this case you should run your script in a non-distributed mode. Please run your script with just `python  finetune.py xxxx` and let us know how it goes<|||||>But , with single GPU , it will take lot and lot of time to finish the training , I want to make sure it uses all my GPU , but its not happening .
I will update the same to you , for single GPU <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.